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ABSTRACT 


ARINC Research Corporation conducted a three-month assessment of the 
National Aeronautics and Space Administration (NASA) Flight Assurance 
Review Program to develop minimum standard guidelines for flight assurance 
reviews. We evaluated mote than 70 documents from eight NASA centers and 
NASA Headquarters to determine current design review practices and 
procedures. On the basis of that analysis, we identified six reviews as 
being the recommended minimum. We analyzed and synthesized the practices 
and procedures used at the different centers to incorporate the most 
effective ones into the minimum standard review guidelines. We defined 
guidelines in terms of procedures, personnel and responsibilities, review 
iteras/data checklist, and feedback and closeout. 

This report presents the results of our assessment. It identifies 
and describes the six reviews recommended and presents the minimum 
standard guidelines developed for flight assurance reviews. It also 
presents observations and conclusions for further improving the NASA 
review and quality assurance process. 
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SUMMARY 


Under Contract NASW-3787, ARINC Research Corporation conducted an 
assessment of the National Aeronautics and Space Administration (NASA) 
Flight Assurance Review Program to develop minimum standard guidelines for 
flight assurance reviews. We identified six reviews as being the 
recommended minimum, and analyzed and synthesized the practices and 
procedures used at the different NASA centers to be incorporated into the 
review guidelines. 

To conduct the assessment, we performed four tasks: 

- Task 1 - Collect and Analyze Data from Centers 

- Task 2 - Develop a Matrix of Flight Review and Certification 

Practices 

- Task 3 - Select the Best Practices 

- Task 4 - Develop Guidelines for NASA-Wide Practices 

During our assessment, we identified the following six reviews as 
being the recommended minimum for the Flight Assurance Review Program: 

- Preliminary Requirements Review 

- Preliminary Design Review 

- Critical Design Review 

- Preshipment Review 

- Design Certification Review 

- Flight Readiness Review 

We defined the minimum standard guidelines for the NASA Flight 
Assurance Review Program in terras of the following items: 
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Procedures 


The method of convening the Review Board or Team, the selection of 
the Board or Team members, and a general description of the activities of 
the Board or Team. 

Personnel and Responsibilities 


Because titles vary from center to center, personnel are described in 
terms of management level and technical competence rather than by title. 
Responsibilities of persons and groups of persons (such as Review Board or 
Team members) are related to the Review Board or Team component (i.e., 
cognizant manager, chairman, etc.). 

Review Items/Data Checklist 


A compilation of matters to be considered by the Review Board or Team 
for each of the six reviews. 

Feedback and Closeout 


A process that begins with the identification of a problem or concern 
and continues through the disposition of all action items, in keeping with 
the basic objective of providing assurance to NASA management that the 
requirements of a program or project are satisfied. 

The minimum standard review guidelines were developed from a 
synthesis of the practices and procedures of the NASA centers so that they 
can be applied to as many projects and programs as possible. Three of the 
preceding items (procedures, personnel and responsibilities, and feedback 
and closeout) are common to all Review Board or Team reviews; the review 
item data checklist is unique to each of the six reviews. 

As a result of our assessment, ARINC Research made the following 
observations: 

- The NASA centers conduct many different reviews to ensure the 
quality and safety of end items. 

- The practices and procedures used in conducting reviews follow 
similar patterns. 

- The terminology used varies somewhat from center to center. 

- The detailed documented descriptions of review practices and 
procedures vary greatly from center to center. Personnel must 
rely on referenced documents to provide the detail lacking in some 
of the documents supplied for this study. 
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- Documents from individual centers do not always clearly describe 
an in-depth review process; it was often necessary to refer to a 
number of documents to obtain confidence that the review process 
has the necessary depth and includes all of the necessary elements. 

- Few of the documents supplied indicate an in-depth feedback and 
closeout process. 

- Some of the documents do not describe procedures of the Review 
Board or Team or a review itera/data checklist. 

- There was little documentation supplied describing the decision 
process for determining payload classifications (A, B, C, or D), 
or for determining the differences in the reviews for each 
classified payload. 

- Host reviews are conducted by the persons directly associated with 
the end item under review. 

- The Review Board chairman is generally directly associated with 
the program or project under review. 

- The review descriptions do not indicate how upper-level Review 
Boards or management are advised of action-item closeouts, leading 
to the assumption that a closeout stops at the level at which it 
was approved. 

- The review descriptions do not indicate a process for moving open 
action items to a higher-level review. 

- The review practices and procedures of the Department of Defense 
(DoD) and the Federal Aviation Administration (FAA) do not offer 
significant areas of improvement to NASA. 

On the basis of these observations, we reached the following 
conclusions: 

- A standard format for review practices and procedures, including 
standard terminology, should be used by all centers. 

- A procedure should be devised and implemented for advising a 
central office at Headquarters of improvements to the standard 
review practices and procedures so that all centers can be kept 
fully informed. 

- Minimum standard review guidelines promulgated by a central office 
at Headquarters should be structured in a manner that allows each 
center flexibility in interpreting the standards to meet diverse 
mission requirements. 
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- A procedure should be devised and implemented for advising a 
central office at Headquarters of changed review practices and 
procedures to meet specific mission requirements. 

- The chairman of the Review Board or Team should be selected first 
for management ability and second for technical competence; the 
chairman should not be directly associated with the program or 
project under review. 

- The number of persons on the Review Board or Team (including the 
chairman) should be an odd number, and should be the minimum 
number necessary to satisfy technical requirements of the program 
or project under review. The majority of the members should be 
persons not directly associated with the program or project under 
review. 

- Members of Review Board or Team should be selected for their 
technical competence and ability to communicate effectively. 

- An approval process should be developed for all changes made under 
the standard flight-critical change program. Approval by a 
minimum of two persons should be required for changes of the 
lowest classification, and approval of a formal review board 
should be required for changes of the highest level of criticality. 

- Each review should begin with a summary of action items from the 
previous review together with a short statement of the current 
disposition of each action item. 

- A specific standard procedure should be developed to evaluate the 
qualification of components, subassemblies, or assemblies on the 
basis of previous applications or similarity to another component, 
subassembly, or assembly. 

- The history of similar systems, components, subassemblies, or 
assemblies should be reviewed at the PDR or CDR so as to identify 
areas of needed concentrated analysis or review to take advantage 
of data regarding previous failures or problem areas. 

- When a design change is approved, it may be necessary to review 
all subsequent changes related to the approved change or made to 
accommodate the approved change. 

- One-person reviews or change approval should be avoided. 

- Standard procedures and methods should be developed for reviewing 
and certifying software. The procedures should include criteria 
for identifying the requirements for recertification of software 
based on either hardware or software changes. 
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- The minimum standard review guidelines presented in this report 
should be considered for implementation as a NASA management 
instruction. 

- The minimum standard review guidelines presented in this report 
should be used as working papers to solicit comments from all 
centers before the guidelines are officially published. A number 
of iterations may be necessary to consolidate and coordinate 
comments from the centers into the final document. 

- An independent third party review may be required for Class A 
payloads if the Review Board guidelines do not provide the 
required freedom of action which should be expected when the 
chairman and over half of the Review Board are not associated with 
the project under review. 

ARINC Research Corporation recommends that the conclusions detailed 
above be considered in the implementation of the minimum standard 
guidelines for the NASA Flight Assurance Program. 
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CHAPTER ONE 


INTRODUCTION 


1.1 BACKGROUND 

One of the functions of the Office of the Chief Engineer of the 
National Aeronautics and Space Administration (NASA) is to provide 
integrated technical and management focus on policies throughout NASA 
concerning design certification reviews of all NASA programs. To fulfill 
this function, the Office began to collect information from the individual 
NASA centers describing the design review processes used. A preliminary 
analysis of the collected information showed that each center has 
established its own practices and procedures for design reviews. 

Different reviews are conducted at different centers and, where there are 
similar reviews at two or more centers, different implementation practices 
are used. Because of these differences, it was determined that a more 
detailed study of the information supplied was required. ARINC Research 
Corporation was contracted to conduct an assessment of the practices and 
procedures used at the different centers, select the most effective ones, 
synthesize them into a set of standard review processes, and develop 
recommended guidelines for their implementation. 


1.2 OBJECTIVES 

The primary objective of the effort was to develop guidelines for 
minimum standard review procedures applicable across all NASA activities. 
The primary objective was to be obtained through completion of two interim 
objectives: 

- Evaluate the existing critical flight program review practices and 
procedures followed by the various NASA centers. 

- Select the most effective practices and procedures used in each 
step of the review process. 


1.3 DESCRIPTION OF TASKS 

The Statement of Work set forth four tasks for the study. They are 
described in the following subsections. 
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1.3.1 Task 1 - Collect and Analyze Data from Centers 


The Office of the Chief Engineer supplied documents from the various 
NASA centers showing current review procedures. The objective of Task 1 
was to analyze the procedures to determine practices followed, data used 
in each review, feedback processes, and the nature of the action item 
review process. 

Each review process was to be charted to compare the various steps 
and decision paths followed at each center, and the charts were to be 
reviewed with the cognizant NASA technical monitor. In addition, a 
glossary of terms used by the different centers was to be compiled to 
provide a basis for establishing common definitions for the practices to 
be followed. 

1.3.2 Task 2 - Develop a Matrix of Plight Review and Certification 
Practices 


The information from Task 1 was to be used to develop matrices for 
the various center reviews showing all of the steps followed, including 
personnel involved; review items/data and analysis required; referenced 
NASA standards, instructions, bulletins, or other documents; and any 
decisions made and possible outcomes. Upon completion of Task 2. a letter 
report was to be prepared documenting the current practices. 

1.3.3 Task 3 - Select the Best Practices 

Following development of the matrices, the most effective practices 
were to be reviewed with NASA and selected in three steps, as follows: 

- Define the steps in the review and design certification process to 
be recommended. 

- Review the matrices to select the best practices applicable. 

- Develop a new matrix for the standard process. 

1.3.4 Task 4 - Develop Guidelines for NASA-Wide Practices 

The results of Task 3 were to be expanded into practices applicable 
throughout NASA for conducting flight assurance reviews. The expanded 
review practices were then to be documented as minimum standard review 
guidelines, defined in terras of: 

- Procedures 

- Personnel and responsibilities 

- Review items/data checklist 

- Feedback and closeout 

In addition, any requirements for new instructions, directives, or 
quality assurance guidelines were to be identified. 
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1.4 ORGANIZATION OF THE REPORT 


Chapter Two of this report presents the study approach, describing 
the analysis of the current review practices and procedures and the 
process by which the minimum standard review guidelines were developed. 
Chapter Three presents the minimum standard guidelines for flight 
assurance reviews. Chapter Four presents the observations and conclusions 
resulting from this study. Appendix A is a glossary of terms; Appendix B 
lists the documents reviewed; Appendix C lists other documents referenced 
by the documents reviewed; Appendix D presents descriptions of current 
review practices; and Appendix E presents a discussion of the applica- 
bility of the minimum standard guidelines to payload classifications A, B, 
C, and D. 
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CHAPTER TWO 


ASSESSMENT APPROACH 


2.1 INTRODUCTION 

Foe our review of current review practices and procedures, the NASA 
Office of the Chief Engineer supplied more than 70 documents from eight 
centers and NASA Headquarters. A list of those documents is presented as 
Appendix B to this report. In general, all the centers described similar 
reviews such as the Preliminary Design Review (PDR) and the Critical 
Design Review (CDR) , but the detailed descriptions of the practices and 
procedures varied greatly from center to center. The differences are 
probably not as great as they appear, however, because many of the 
documents reference other documents (not supplied) that may include 
additional procedural details. The documents reviewed provided us with 
sufficient data to develop guidelines for agency-wide reviews. 


2.2 ANALYSIS OF CURRENT PRACTICES AND PROCEDURES 

We analyzed the supplied documents in four steps. 

We first read each document and developed an annotated flow chart for 
each review described. Figure 2-1 is a sample flow chart. We also 
prepared a list of terms (Appendix A) and a list of other documents 
referenced in the documents under study (Appendix C) . 

The second step was the development of a matrix that showed, for each 
described review, procedures, personnel and responsibilities, review 
items/data checklist, and feedback and closeout. Figure 2-2 is a sample 
matrix. Each matrix was headed by the title and purpose of the review, 
and keyed to the applicable NASA center and document. The appropriate 
sections from the document under study were copied and pasted in the 
indicated matrix column (see Appendix D) . 

In the third step, we identified six reviews as being the recommended 
minimum for the flight assurance review program: 

- Preliminary Requirements Review (PRR) 

- Preliminary Design Review (PDR) 
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Figure 2-1. SAMPLE REVIEW FLOW CHART 



























- Critical Design Review (CDR) 

- Preshipment Review (PSR) 

- Design Certification Review (DCR) 

- Flight Readiness Review (FRR) 

We prepared a consolidated matrix for each of these reviews from the 
matrices developed in Step 2. Figure 2-3 illustrates the matrix form. In 
addition to reviewing the NASA documents, we also analyzed both Department 
of Defense (DoD) and Federal Aviation Administration (FAA) design review 
documents and interviewed persons familiar with the DoD and FAA review 
procedures to determine if any practices and procedures were applicable to 
NASA. 


Finally, we analyzed the six consolidated matrices and the prepared 
flow charts to establish the following: 

- Purpose of each of the six reviews 

- Procedures to be followed in setting up or convening the Review 
Board or Team 

- The personnel involved and their responsibilities in the review 
process 

- A comprehensive review items/data checklist for each of the six 
reviews 

- A feedback and closeout process common to all reviews 

Our original purpose had been to select the best review practices and 
procedures from among those described. In our analysis, we found that 
this was not evident. Practices and procedures of the various centers 
were similar, however, and some were described in more detail than others 
which resulted in the assembly of composite practices and procedures which 
appeared to be most effective in terras of NASA-wide guidelines. 


2.3 DEVELOPMENT OF MINIMUM STANDARD GUIDELINES 

To develop the minimum standard guidelines for the six reviews, we 
studied the six consolidated matrices — one for each type of review. The 
guidelines for each type of review were also related to the standard 
payload classifications. (Payload classifications are described in 
Section 3.5.) 
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DESIGN CERTIFICATION REVIEW 



Figure 2-3. CONSOLIDATED REVIEW MATRIX 


















The procedures for convening a Review Board or Team were defined as 
being common to all reviews. Personnel and responsibilities of the Review 
Board or Team were also considered to be common to all reviews. A third 
area of commonality was the feedback and closeout process, the main 
purpose of which is to inform all personnel of the disposition of all 
action items established during the review process. 

The review items/data checklist was determined to be unique to each 
type of review. The checklists were compiled from the various lists of 
items and data to be reviewed as described in the documents supplied. 
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CHAPTER THREE 


MINIMUM STANDARD GUIDELINES FOR THE 
NASA FLIGHT ASSURANCE REVIEW PROGRAM 


3 . 1 INTRODUCTION 

Our analysis of documents from various NASA centers indicated that a 
number of similar reviews are conducted throughout the agency. Although 
the processes and terminology used vary from center to center and. in some 
cases, within the same center, reviews with similar titles are conducted 
in somewhat similar fashion. We found this similarity to be sufficient 
for the development of agency-wide review guidelines. 

The minimum standard review guidelines described in this chapter are 
intended to serve as an aid in establishing an agency-wide Flight 
Assurance Review Program. The basic objective of the program is to 
provide assurance to management that a design satisfies the specified 
requirements for operations, hardware and software end items, and systems 
for all mission stages, from original definition through successful flight. 

The Flight Assurance Review Program consists of a series of design 
reviews, during which applicable technically competent persons convene to 
determine whether an end item meets specified requirements. The 
objectives of the review program include the following: 

- To evaluate whether a design meets the specified end item 
requi reraents . 

- To evaluate whether a design meets the specified requirements of 
an assembly of end items and of a total system end item. 

- To identify problems and devise solutions to problems identified 
for all mission stages, from original definition through flight, 
including the areas of design, production, assembly, tests, and 
operations. 

- To evaluate the effects that proposed design and configuration 
changes have on performance. 
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- To evaluate the designed operational capabilities of an end item 
in teems of state-of-the-art technology. 

- To determine whether specifications satisfy mission requirements. 

- To determine whether the design of an end item is optimized within 
its specified functional requirements. 

- To determine the effect that a change in end item design has on 
the baseline configuration. 

The design reviews are conducted by Review Board or Teams. Section 

3.2 describes the levels of Review Boards. 


3.2 LEVELS OF REVIEW BOARDS 

The Review Board can operate on any of several levels, depending on 
the complexity, criticality, and visibility of the program, project, 
system, or task under review. The Level I review, the highest level of 
review, is conducted by an associate administrator at NASA Headquarters as 
a total system review; the Level II review is conducted at the cognizant 
center management level as a program review; the Level III review is 
conducted as the project management level as a project review; and Level 
IV review is conducted at the task management level as an operations, 
hardware, or software end item review. 


3.3 MINIMUM STANDARD REVIEW GUIDELINES 

In determining the minimum standard guidelines for the Flight 
Assurance Review Program, we defined each review in terms of procedures, 
personnel and responsibilities, review items/data checklist, and feedback 
and closeout. All of these areas except review items/data checklist are 
generic to each of the six reviews identified: the review items/data 
checklist is unique to each review. The four areas are addressed in the 
following subsections. Guidelines for procedures, personnel and 
responsibilities, and feedback and closeout are presented in Sections 
3.3.1 through 3.3.3, respectively. Section 3.3.4 briefly describes the 
six reviews and presents the review items/data checklist guidelines for 
each. 

3.3.1 Procedures 


Procedures include the method of convening the Review Board, the 
selection of Board members, and a general description of the activities of 
the Review Board. 
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3. 3. 1.1 Convening of Review Board 


Figure 3-1 illustrates the organization of the Review Board. A 
Review Board should be convened by the convening authority which is the 
in-line management at least one level above the program or project under 
review. The responsibilities of the convening authority should include 
the following: 

- Prepare and issue a Review Board charter identifying the purpose, 
scope, objectives, and authority of the Review Board. 

- Identify the specific functions of the Review Board. 

- With the assistance of the cognizant manager, select a Review 
Board chairman. 

- Review agenda and schedule. 

- Review Board conclusions and recommendations. 

- Approve or reject Board conclusions and recommendations. 

The convening authority should be assisted by the cognizant manager 
of the program, project, system, or task under review. The cognizant 
manager should assist the convening authority and support the Review Board 
as follows: 


- With the concurrence of the convening authority, recommend the 
Review Board chairman. 

- With the assistance and concurrence of the Review Board chairman, 
recommend the Review Board members. 

- Assist the Review Board chairman in the development of the review 
agenda in accordance with the charter. 

- Ensure that the review item/data package adequately supports the 
Review Board charter. 

- Prepare a description of how action item disposition will be 
handled for the convening authority. 

- Monitor the status of action items and report closeouts to the 
convening authority. 

- Approve and impose the generated action items. 
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CONVENING 

AUTHORITY 



Figure 3-1. REVIEW BOARD ORGANIZATION 






















3. 3.1. 2 Membership of Review Board 

The Review Board chairman selected must have the technical competence 
to understand the general technical aspects of the program, project, 
system, or task under review. Most important, he or she should have the 
ability to maintain a favorable review climate by fostering free 
discussion. It is preferable that the chairman not be directly associated 
with the program, project, system or task under review in order to 
eliminate any tendency toward bias. 

Board members should be selected on the basis of their individual 
technical capabilities regarding the review subject (s). so that the 
collective capabilities and disciplines will satisfy the review 
objectives. Preferably, the members should have broad experience in 
different projects. In the event that a permanent Review Board is 
established to function over an extended period, additional "standby" 
members should be selected for their recognized capabilities in the 
required specialities and should be rotated as appropriate. 

The number of Review Board members should not exceed the minimum 
necessary to encompass all the required disciplines under review. The 
majority of the Review Board, including the chairman, should be members 
who are not directly associated with the program, project, system, or task 
under review. 

The Review Board often will be supported by teams and subteams 
directly involved in the review subject (s). In some cases the Review 
Board will require the support of a specialist team, which may be 
assembled from the NASA center, from other centers, or from outside NASA. 
Often it may be most effective if the specialist team is composed mainly 
of individuals who have the necessary technical knowledge but are not 
directly associated with the program, project, system, or task under 
review. 

3.3. 1.3 Activities of Review Board 

As previously stated, the purpose of the Review Board is to approve 
an end item and to provide to NASA management and contractors assurance 
that the most satisfactory design has been selected, that an end item has 
been produced to meet the specified requirements, or that an end item is 
ready for flight. The Review Board is convened for a single purpose or 
for multiple purposes, as required by its charter. Input to and output 
from the Review Board consists of a systeraized and disciplined set of 
documents as well as oral presentations describing concerns, problems, 
requests for actions, and the disposition of action items. 
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Input to the Review Board may include a few requests for action, 
review items, or concerns, or it may constitute a large number of items. 
The initial input document should be a single-page form describing each 
request for action, review item, or concern with supporting 
documentation. The document should be delivered as a review package to 
the Board, the convening authority, and the cognizant manager in advance 
of the scheduled meeting. All personnel should review the review package 
before the meeting. 

Action items constitute a significant output of the Review Board. 
Action items are documented on a standardized form and assigned, with the 
assistance of the cognizant manager, to an assignee for action. The 
assignee may accept or reject the action item, but in either case the 
standardized form must be completed and returned to the Review Board with 
supporting documentation and data as required. 

The deliberations of the Review Board should be conducted in an open 
forum during which all considerations (approval or rejection) of request 
for action or review items and concerns are voiced and recorded. The 
audience should be free to comment. The conclusions of the Review Board, 
whether arrived at unanimously or by consensus, should be documented, 
together with the Board minutes and any dissenting opinion, for the record 
and for distribution to the convening authority, cognizant manager, and 
Board. 


3.3.2 Personnel and Responsibilities 

Inasmuch as the members of the Review Board are selected on the basis 
of their collective capabilities and disciplines, the persons assigned 
need not be the leading authorities in their respective specialities. 
However, it is important that the collective technical competency of the 
Review Board be comparable to that which produced the design under 
review. It is also important that the Board include the participation of 
as many individuals as possible that are not directly associated with the 
program or project under review. Further, the Board must be able to 
effectively communicate its deliberations. Therefore, the Review Board 
must include the proper mix of personnel with the technical skills and 
communications capability to effectively document the results of the 
review. 

In addition to being responsible for directing program or project 
management's attention to design deficiencies, the Review Board must 
ensure that a favorable climate exists, including adequate time for 
preparation, and that the review procedures provide for free discussion. 
The Review Board must not establish a climate that would tend to penalize 
a person presenting "bad news." 
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The specific responsibilities of each component of the Review Board 
are as follows: 

- Convening Authority 

— Confirm the classification of the payload end item. 

— With the assistance of the cognizant manager, select the 
review board chairman. 

— Issue the charter describing the purpose, scope, and authority 
of the review. 

— Report the findings of the Board to the next- level review as 
appropriate, including a summary of action item dispositions. 

— Approve the cognizant manager's response to the Review Board 
report . 

— Transmit the summary of action item dispositions and open 
action item documentation and data to the next-level review. 

- Cognizant Manager 

— Recommend candidates for Review Board chairman to the 
convening authority. 

— With the assistance of the chairman select the Review Board 
members. 

— Contribute to the development of the Review Board charter. 

— Ensure that the review package is adequate and distributed on 
schedule . 

— Support the Review Board chairman in developing the review 
agenda . 

— Recommend to the convening authority disposition of the 
findings and recommendations of the Board. 

— Approve and assign the action items that are generated. 

— Prepare for the convening authority’s approval the written 
response to the Board's report, including how action items 
will be handled. 

— Monitor the status of action items and report the closeout of 
action items to the convening authority. 
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Review Board Chairman 


— With the assistance of the cognizant manager, establish a 
Review Board agenda to meet the charter requirements, 
including all information relevant to the charter. 

— Assist the cognizant manager in the selection of the Review 
Board members. 

— Specify the date and location of review meetings. 

— Appoint a secretary. 

— Chair the review and keep the discussion open, on the subject, 
and on schedule. 

— Convene the Review Board as required to deliberate, and to 
arrive at findings and recommendations. 

— Develop, where necessary, a consensus of findings of the 
Board, including an assessment of the risk associated with 
problem areas; develop recommendations for action. 

— Ensure that a written report of findings, including 
recommendations for actions, is submitted on a timely basis to 
the convening authority, with a copy to the cognizant manager. 

— Approve action item closeout before distribution to Board 
members. 

Review Board Members 

— Become familiar with the documentation and presentation 
material and with the objectives and criteria for assessment 
and evaluation before the review. 

— Participate in all scheduled meetings and contribute to the 
fulfillment of the objectives of the Review Board. 

— Assist in developing a disposition for the material reviewed, 
make recommendations for action, and contribute to the Review 
Board report. 

Review Board Secretary 

— Record comments of the Board members. 

— Assist the chairman in preparing the Board report. 

— Maintain a file of all Board actions, including closeout 
memoranda. 
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- Review Board Secretary (continued) 

— Forward action items to Action Item Control and assignee(s). 

— Provide signature approval of the Board report and prepare any 
minority report. 

- Presenters to the Review Board 

— Assemble the data package for the review, including 

appropriate documents, and provide handouts of the oral 
presentation material and supplementary data as required. 

— Make the oral presentation and respond to questions. 

— Assist the cognizant manager in defining action items. 

— Follow up on and respond in writing to action items assigned. 

Action Item Control may be an assigned individual, a group of 
individuals, or a permanent organization, depending on the complexity of 
the program, project, system, or task under review. The responsibilities 
of Action Item Control are as follows: 

- Assign and maintain action item control numbers for each action 
item. 

- Maintain the action item and disposition file. 

- Distribute numbered action items to related action assignees. 

- Prepare an action item summary report showing action item number, 
action item description, disposition, and closeout (or open), and 
distribute it to the convening authority, cognizant manager. 

Review Board members, and action assignee before the next Review 
Board meeting. 

3.3.3 Feedback and Closeout 


The major purpose of the feedback and closeout process is to ensure 
that the convening authority, cognizant manager. Review Board chairman, 
and members are kept fully apprised of the status of all closed and open 
action item reports (also called Review Item Disposition or RID reports). 
Figure 3-2 illustrates the feedback and closeout process. 

The convening authority convenes the Review Board on the basis of 
action item requests, problems, and concerns brought to his or her 
attention. In continuing the Review Board, the convening authority 
approves or rejects Review Board actions and approves/certifies Review 
Board reports to the next- level review. The Cognizant Manager is kept 
fully apprised of all activities of the convening authority and Review 
Board and supports both as required. The cognizant manager may return a 
rejected request for action, problem, concern, open or closed action item 
to the convening authority or the Review Board for review. 
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CLOSED 


CLOSED 


Figure 3-2. FEEDBACK AND CLOSEOUT FLOW CHART 
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The Review Board chairman, by direction to the secretary, documents 
all Board actions and sends copies to all members, the convening 
authority, and the cognizant manager. The action item (RID) output of the 
Review Board is sent to Action Item Control and is assigned to the 
appropriate team or subteam for action with the concurrence of appropriate 
management. The assignee (s) within the team or subteara document the 
actions taken and inform Action Item Control and assignee management, who 
in turn issue open and closed RID reports, with supporting documentation, 
to the Review Board. This process continues in a loop until all RIDs are 
closed, or until the convening authority determines that remaining open 
items must be elevated to another convening authority for a higher-level 
Review Board. 

The feedback and closeout process is a fundamental audit trail of 
action item requests, problems, concerns, and potential problem areas. 

Each center should develop specific procedures and implementation 
directives that clearly define the handling and disposition of open and 
closed RIDs to ensure that the action items are treated properly and that 
mechanisms are established to ensure sufficient visibility of the results 
of the disposition of the items. 


3.4 FLIGHT ASSURANCE REVIEWS AND REVIEW ITEMS/DATA CHECKLISTS 

The review iteras/data checklist consists of matters to be considered 
by Review Board for each review. As explained previously, the checklists 
are unique to each of the six reviews identified. The following 
subsections present a brief description of the reviews, together with the 
review iteras/data checklist for each. Individual reviews may be modified 
by charter to meet the requirements of the program, project, system, or 
task under review. 

3.4.1 Preliminary Requirements Review (PRR) 

The purpose of the PRR is to establish the requirements of and the 
conceptual approaches to meeting the requirements of the program, project, 
system, or task. The PRR lays the groundwork for ensuing reviews and 
establishes the following: 

- Confirmation of the payload classification as A, B, C, or D (see 
Section 3.5) 

- Verification of the configuration concepts and requirements 

- Verification of the mission objectives 

- Qualification of the conceptualized approach 

- Approval of the requirements baseline for program or project 

- Confirmation that the functional requirements for the system and 
subsystem are responsive to the end-item objectives and constraints 
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PRR purposes (continued) 

- End item suitability of the selected configuration by reference to 
preliminary drawings and study reports 

- Expected suitability of the end item baseline configuration to 
meet the required schedule 

- Feasibility and development tests required to select and 
substantiate design approaches 

The PRR review items/data checklist includes the following: 

- Statement of mission objectives 

- Analyses of mission requirements 

- Preliminary preferred requirements 

- Functional analyses 

- Preliminary interface requirements 

- Support agreements 

- Preliminary reliability and quality assurance requirements 

- Scientific requirements 

- Concept drawings and sketches 

- Block and logic diagrams 

- Costs and schedules 

- Design-concept documentation 

- General test requirements 

- Preliminary study reports 

- Heritage analyses 

3.4.2 Preliminary Design Review (PDR) 

The purpose of the PDR is to establish the design and ensure that it 
meets the program, project, system, or task end item baseline 
requirements. The PDR process includes approval of the basic design 
approach, confirmation of the design requirements, and evaluation of the 
progress and technical adequacy of the design approach. Further, it 
determines whether the selected design approach is compatible with the end 
item. 
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The PDR review items/data checklist includes the following: 

- Status of PRR action items 

- Final functional requirements 

- Analyses for compliance with functional requirements and 
specifications 

- Analyses of suitability of inherited designs and hardware 

- Environmental design requirements 

- Interface design requirements 

- Design parameters, restraints, and constraints 

- Block diagrams 

- Single-point failures analyses 

- List of preliminary parts, materials, and processes 

- Plan and schedule for hardware and software development (including 
verification tests on analyses to be performed) 

- Software standards to be applied 

- Experiment performance analysis 

- Preliminary data management flow and reduction plans 

- Plans for safety, reliability, and quality assurance compliance 

- Plans for spares provisioning 

- Requirements and plans for support equipment 

- Plans for preliminary operations 

- Schedules and status of the project, including cost and technical 
developments 

- Design optimization analyses 

- Plans and controls documentation 

- Design traceability 
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3.4.3 Critical Design Review (CDR) 

The purpose of the CDR is to verify, when the design of an end item 
is essentially complete, the suitability of the design in meeting the 
specified requirements, and to determine whether the end item is ready for 
manufacture. 

The CDR determines whether the designed end item is compatible with 
the specified requirements, and verifies that the design conforms to the 
requirements established at the PDR and updated to the time of the CDR. 
During the CDR, the integrity of the design is verified through review of 
analytical and test data and the system capability is established through 
reference to all system engineering documentation. The CDR approves or 
rejects all released drawings or drawings ready for release. 

As a result of the CDR, the design baseline is established, end item 
specifications are updated, and specific end item designs are accepted for 
release for fabrication. 

The CDR review items/data checklist includes the following: 

- Status of PDR action items 

- Assessment of hardware and software inheritance 

- Test and performance specifications 

- Prototype test results 

- Design mechanization and analysis 

- Design trade-offs and alternatives considered 

- Detailed interfaces and cable configurations 

- Detailed analysis of failure modes 

- Analyses of safety and human factors 

- Maintainability, repairability, operability, and reliability 
considerations 

- Schedule and resource plans 

- Man-machine interfaces 

- Training plans 

- Conformance of the design to the functional requirements 

- Differences between the system and subsystem performances in 
relation to the performances estimated at the Preliminary Design 
Review 
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CDR review items/data checklist (continued) 

- Detailed design parameters, restraints, and constraints 

- Interface details and agreements 

- Weight and power 

- Detailed circuit drawings 

- Electronic parts classification 

- Screening specifications 

- List of nonelectronic parts, materials, and processing 

- List of purchased devices 

- Materials and processing specifications 

- Identification and traceability of materials and specifications 

- Electronic and mechanical parts stress analysis results 

- Worst-case analysis of end-item failure 

- Conformance to environmental design requirements 

- Details of design, construction, and electronics packaging 

- Quality assurance plans and procedures 

- Manufacturing plans 

- Configuration control plans (hardware and software) 

- Documentation status 

- Flow plan and schedule status (hardware and software) 

- Problem/ failure reporting system 

- Failure modes and effects analysis 

- Preliminary test plans 

- Developmental and detailed test results 

- Qualification and acceptance test plans 

- Calibration plan 

- Data management plan 
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CDR review items/data checklist (continued) 

- Formal and informal report plans 

- Final design verification plans 

- Constraints and precautions for initial user handling or operation 

- Spares provisioning 

- Requirements for support equipment 

- Flight operations approach 

- Schedule and cost status 

- Risk assessment 

3.4.4 Preshipment Review (PSR) 

The purpose of the PSR is to verify that testing is complete with no 
unresolved problems, to evaluate the readiness of the end item for 
delivery, and to verify the readiness of ground-based facilities and the 
integrated flight team. 

The PSR review iteras/data checklist includes the following: 

- Status of previous action item closeouts 

- Current approved drawings and specifications, and adequate 
definition of the end item, including approved specification 
changes and waivers 

- Status and documentation of the end item, including all interface 
documentation, to ensure that the latest changes have been 
incorporated in affected hardware 

- Status of test documentation and determination (by review of the 
test results) of the adequate and proper testing of the end item 

- Documentation to determine the status of the problem/ failure 
report and evaluate its total impact, particularly with respect to 
risk for the mission 

- Documentation of the quality assurance history of the end item to 
ensure that necessary inspections have been performed and that all 
discrepancies have been properly closed out 

- Requirements and plans for shipping and handling 

- Verification of compliance with safety requirements 

- Status and acceptability of the deliverable end-item data package 

- Risk assessment 
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3.4.5 Design Certification Review (DCR) 

The purpose of the DCR is to ensure that the end item has reached 
design and developmental maturity through reexamination of the 
requirements in light of NASA goals and to certify that the entire system 
design meets those requirements. The DCR is always a Level I review, is 
chaired by an Associate Administrator, and includes all aspects of the 
program — hardware and software, ground and flight operations, and 
personnel (e.g., training, risk/contingency, and safety). The DCR 
assesses and certifies the design of a hardware or software end item or a 
space vehicle for flight worthiness and flight safety, and assesses and 
certifies the design of the launch complex, the mission control center, 
the space flight network, and launch instrumentation for the associated 
mission. 

The DCR review iteras/data checklist includes the following: 

- Status of previous action items 

- Item change data 

- Analysis and test documentation 

- Reliability documentation 

- Analyses of safety and human factors 

- Specified requirements derived from mission objectives 

- Analysis of previous failures and incidents 

- Verification of physical and functional configuration 

- Validation of analytical models of the equipment and system 

- Validation of the thermal-control design 

- Review of hazards control 

- Checklist of a safety systems and subsystem including the 
following: 

— Electrical 

— Materials 

— Mechanical 

— Pressure 

— Propulsion 
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DCR review items/data checklist (continued) 


— Pyrotechnics 
— Structures 
— Cautions and warnings 
— Environmental control 
— Cryogenics 
— Hydraulics 
— Optics 
— Human factors 

- Training of ground flight personnel 

- Operational plans and status 

- Mission rules 

- Results and status of verification planning, testing, and analysis 
for the total system and associated ground support systems 

- Deviations from baseline configuration 

- Configuration inspection documentation 
3.4.6 Flight Readiness Review (FRR) 

The purpose of the FRR is to assess the overall readiness of the 
total system and the flight, including readiness to achieve all flight 
objectives. 

The FRR review items/data checklist includes the following: 

- Status of previous action item closeouts 

- Status of vehicle and launch support facilities, systems, 
equipment, range support, launch commit criteria, and personnel 
training 

- Reports on residual hazards and problems 

- Status of operation and maintenance instructions 

- Verification of completion of documentation, including the 
following: 

— Structures 
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FRR review items/data checklist (continued) 

— Dynamic coupling 

— Integrated vehicle control capability 
— Staging 
— Pyrotechnics 
— Safe mission termination 
— Radio frequency transmissions 
— Weight and center-of-gravity control 
— Baseline mission capability 
— Payload support requirements 

— Ground support equipment and facility safety design 
— Ground operations control and monitor capability 

— Hydraulic power capability 

/ 

— Emergency egress design capability (as applicable) 

— Landing station capability 
— Range safety system 
— Communications and tracking 

- Configuration changes since the previous review 

- Hardware and software anomalies 

- Limited-life components, life remaining, and age life/time cycle 

- Status of logistics readiness (spares support) 

- Identification of actual hardware shortages and open work items 

- Assessment of safety, reliability, and quality assurance 

- Identification of problems that may constrain the flight 

- Failures/ incidents and accidents reports 
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3.5 PAYLOAD CLASSIFICATIONS 


The Review Board's charter should include the classification of any 
payload under review. The classification should be determined on the 
basis of NASA Management Instruction NMI 8010.1, Classification of NASA 
Space Transportation System (STS) Payloads. That document states the 
classification of payloads as follows: 

- Class A: Minimum Risk . Payloads for which a minimum 

risk approach is clearly dictated by prohibitively high 
cost of the consequences of failure, or by an 
unacceptable combination of costs and intangible factors 
associated with failure. 

- Class B: Risk/Cost Compromise . Payloads for which an 

approach characterized by reasonable compromise between 
minimum risks and minimum costs is appropriate due to 
the capability to recover from in-flight failure by some 
means that is marginally acceptable, even though it 
involves significantly high costs and/or highly 
undesirable intangible factors. 

- Class C: Economically Reflvable or Repeatable . 

Payloads for which reflight or repeat flight is planned 
as a routine back-up in the event of in-flight soft 
failure, and reflight or repeat flight costs are low 
enough to justify limiting qualification and acceptance 
testing to end-item environment screening. (In addition 
to whatever is required for STS safety and compatibility 
and payload functional testing.) There is no 
significant intangible or tangible impact of soft 
failure except the cost of repair and reflight, or 
repeat flight which can be estimated with reasonable 
confidence and can be directly related to 
flight-reliability enhancement costs. Therefore, a 
decision criteria of minimum total expected cost is 
appropriate and practical. 

- Class D: Minimum Single Attempt Cost . Payloads that 

have objectives worth achieving at a cost not-to-exceed 
the amount required for a single low-cost attempt where 
formal verification requirements are limited to those 
necessary for safety and compatibility. 

A specific payload should be classified as A, B, C, or D as early as 
the payload project definition, but no later than the Preliminary 
Requirements Review. Early classification of the payload is necessary to 
properly apply the NMI 8010.1 requirements. Proper application of the 
classification is required to ensure that the end item (payload) will meet 
the design specifications at the lowest cost consistent with the stated 
level of confidence for the specific end item. 
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Headquarters Program Offices are responsible for assigning the 
initial payload classifications, and field installations are responsible 
for recommending classifications for payloads and payload assemblies and 
subassemblies within their area of responsibility. Because NMI 8010.1 is 
highly project-oriented, it is appropriate for each center to review this 
management instruction in light of individual mission or project 
responsibilities. Appendix E presents a discussion of the applicability 
of the minimum standard guidelines to payload classifications A, B, C, and 
D. 
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CHAPTER FOUR 


OBSERVATIONS AND CONCLUSIONS 


4.1 OBSERVATIONS 

As a result of our assessment, ARINC Research made the following 
observations: 

- The NASA centers conduct many different reviews to ensure the 
quality and safety of end items. 

- The practices and procedures used in conducting reviews follow 
similar patterns. 

- The terminology used varies somewhat from center to center. 

- The detailed documented descriptions of review practices and 
procedures vary greatly from center to center. Personnel must 
rely on referenced documents to provide the detail lacking in some 
of the documents supplied for this study. 

- Documents from individual centers do not always clearly describe 
an in-depth review process; it was often necessary to refer to a 
number of documents to obtain confidence that the review process 
has the necessary depth and includes all of the necessary elements. 

- Few of the documents supplied indicate an in-depth feedback and 
closeout process. 

- Some of the documents do not describe procedures of the Review 
Board or Team or a review itera/data checklist. 

- There was little documentation supplied describing the decision 
process for determining payload classifications (A, B, C, or D) 
and determining the differences in the scope of the reviews for 
each classified payload. 

- Many reviews are conducted by the persons directly associated with 
the end item under review. 
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- The Review Board chairman is generally directly associated with 
the program or project under review. 

- The review descriptions do not indicate how upper-level Review 
Boards or management ate advised of action item closeouts, leading 
to the assumption that a closeout stops at the level at which it 
was approved. 

- The review descriptions do not indicate a process for moving open 
action items to a higher-level review. 

- The review practices and procedures of the DoD and FAA do not 
offer significant areas of improvement to NASA. 


4.2 CONCLUSIONS 

On the basis of our observations, we reached the following 
conclusions: 

- A standard format for review practices and procedures, including 
standard terminology should be used by all centers. 

- A procedure should be devised and implemented for advising a 
central office at Headquarters of improvements to the standard 
review practices and procedures so that all centers can be kept 
fully informed. 

- Minimum standard review guidelines promulgated by a central office 
at Headquarters should be structured in a manner that allows each 
center flexibility in interpreting the standards to meet diverse 
mission requirements. 

- A procedure should be devised and implemented for advising a 
central office at Headquarters of changed review practices and 
procedures to meet specific mission requirements. 

- The chairman of the Review Board or Team should be selected first 
for management ability and second for technical competence; the 
chairman should not be directly associated with the program or 
project under review. 

- The number of persons on the Review Board or Team (including the 
chairman) should be an odd number, and should be the minimum 
number necessary to satisfy the technical requirements of the 
program or project under review. The majority of the members 
should be persons not directly associated with the program or 
project under review. 

- Members of the Review Board or Team should be selected for their 
technical competence and ability to communicate effectively. 
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- An approval process should be developed for all changes made under 
the standard flight-critical change program. Approval by a 
minimum of two persons should be required for changes of the 
lowest classification, and approval of a formal Review Board 
should be required for changes of the highest level of criticality. 

- Each review should begin with a summary of action items from the 
previous review together with a short statement of the current 
disposition of each action item. 

- A specific standard procedure should be developed to evaluate the 
qualification of components, subassemblies, or assemblies on the 
basis of previous applications or similarity to another component, 
subassembly, or assembly. 

- The history of similar systems, components, subassemblies, or 
assemblies should be reviewed at the PDR or CDR so as to identify 
areas of needed concentrated analysis or review to take advantage 
of data regarding previous failures or problem areas. 

- When a design change is approved, it may be necessary to review 
all subsequent changes related to the approved change or made to 
accommodate the approved change. 

- One-person reviews or change approval should be avoided. 

- Standard procedures and methods should be developed for reviewing 
and certifying software. The procedures should include criteria 
for identifying the requirements for recertification of software 
based on either hardware or software changes. 

- The minimum standard review guidelines presented in this report 
should be considered for implementation as a NASA management 
instruction. 

- The minimum standard review guidelines presented in this report 
should be used as working papers to solicit comments from all 
centers before the guidelines are officially published. A number 
of iterations may be necessary to consolidate and coordinate 
comments from the centers into the final document. 

- An independent third party review may be required for Class A 
payloads if the Review Board guidelines do not provide the 
requited freedom of action which should be expected when the 
chairman and over half of the Review Board are not associated with 
the project under review. 

ARINC Research Corporation recommends that the conclusions detailed 
above be considered in the implementation of the minimum standard 
guidelines for the NASA Flight Assurance Program. 
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APPENDIX A 


GLOSSARY OF TERMS 


A glossary of standard terms should be included as an integral part 
of the minimum standard guidelines for the Flight Assurance Review program. 
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GLOSSARY OF TERMS 


Associate Administrator, Space 
Transportation System 

The act of an authorized representative of 
the Government by which the government 
assents to its ownership of existing and 
identified articles or approves specific 
services rendered as partial or complete 
performance of the contract. 

Acceptance Test: The environmental, electrical, mechanical, 

functional, and other tests that payloads, 
spacecraft, subsystems, instruments, and 
components scheduled for flight must pass 
before launch. These tests are planned to 
approximate the expected flight 
conditions. They are designed to detect 
nonconformances in material and 
workmanship. 

Accident/Incident : An unplanned event that results in 

personnel fatality or injury or damage to 
or loss of equipment, environment, public 
property, or private property or that 
could result in an unsafe situation or 
operational mode. An accident refers to a 
major event whereas an incident is a minor 
event or episode that could lead to an 
accident . 

Action Assignee: The individual or group given the 

responsibility for completing a specific 
action item. 

Action Item: A specifically defined problem or set of 

problems requiring resolution as 
determined by a review board or review 
team. 

ADP: Acceptance Data Package 

Aerospace Ground Equipment: All equipment required on the ground to 

make an aerospace system operational in 
its intended environment. 

AGE : Aerospace Ground Equipment 

ALT: Approach and Landing Test 


AA-STS: 

Acceptance: 
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GLOSSARY OP TERMS (contd.) 



Article: 


Assembly 


AR: 

ASE: 

Associate Contractor: 


ASSY: 

ATE: 


Baseline Confiquratlon: 



A process used in lieu of, or in addition 
to, testing to validate compliance of an 
item to specification requirements. The 
techniques include the use of analytical 
models in systems engineering analysis, 
statistics and qualitative analyses, 
safety analyses, computer and hardware 
simulations, and analog modeling. 

A unit of hardware or any portion thereof 
required by the contract. 

A functional subdivision of a component, 
consisting of parts or subassemblies that 
perform functions necessary for the 
operation of the component as a whole 
(e.g., regulator assembly, power amplifier 
assembly, gyro assembly, primary 
structure, boom, and solar array). 

Acceptance Review 

Airborne Support Equipment 

The contractor who under direct contract 
to NASA performs work excluded from the 
principal contract. The associate 
contractor is responsible to the principal 
contractor for technical integration of 
the (sub) system and must coordinate 
technical developments and requirements in 
a timely and organized manner. The 
associate contractor is directly 
responsible to NASA for administrative and 
contractual matters. 

Assembly 

Automatic Test Equipment 

The documented and approved design concept 
or arrangement of components as 
established at a given point in the 
procurement cycle for systems or equipment. 

A line drawing with block outlines to 
designate units or functional groups for 
general arrangement studies, functional 
explanation, product familiarization, etc. 
within a system, set, or item. 
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GLOSSARY OF TERMS (contd.) 


Breadboard: 


CAR: 


Cargo: 


Catastrophic Hazard: 


CCB: 


An assembly of preliminary circuits or 
parts used to prove the feasibility of a 
device, circuit, system, or principle 
without regard to the final configuration 
or packaging of the parts. 

Configuration and Acceptance Review; 
Corrective Action Request. 

Everything contained in the shuttle 
payload bay, plus other equipment located 
elsewhere in the orbiter that is payload- 
unique and not carried in the standard 
baseline orbiter weight budget, including 
payloads and payload support equipments. 

A hazard with the potential for personnel 
injury or resulting loss of life or for 
preventing the safe return to Earth of an 
orbiter vehicle. 

Configuration Control Board 


CCP: 


Configuration Control Panel 


CDDT: 


Countdown Demonstration Test 


CDR: Critical Design Review 

CEI: Contract End Item 

Certificate of Flight 

Worthiness: The principal record at key milestone 

events of an end item's progress. 

Certification of flight worthiness 

provides the means by which centers, 
center end item contractors and NASA Using 
Sites certify to end item configuration 
and design drawings and specifications, 
availability and disposition of required 
documentation and compatibility of end 
item configuration, launch and orbit 
operations documentation. 


A formal documented approval through the 
safety assessment procedure. Includes a 
statement that all safety requirements of 
NHB 1700.7 have been met or, if not, what 
waivers are applicable. 


Certificate of Safety 
Compliance: 
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GLOSSARY OF TERMS (contd.) 


Certification: The approval of an end item, assembly of 

parts, soft-ware or system confirming that 
the specification requirements have been 
met in all respects. 

Checklist : A list of procedures or items summarizing 

the activities required for an operator or 
technician to perform his duties; a 
condensed guide; an on-the-job supplement 
to more detailed job instructions. 


Cl: 


Configuration Inspection 


CITE: 


Cargo Interface Test Equipment 


COFR: Certification of Flight Worthiness 

COFW: Certification of Flight Readiness 

Cognizant NASA Installation: That major organizatioal unit of NASA 

which has direct technical and managerial 
responsibility for the System under 
contract . 


Component : 


Conceptual Phase: 


Configuration: 


Configuration Control: 


A combination of parts, subassemblies, or 
assemblies, usually self-contained, which 
performs a distinctive function in the 
operation of the overall equipment; a 
"black box" 

That period in the system life cycle which 
usually terminates with publication of a 
specific operational requirement. 

The technical and physical description 
required to fabricate, test, accept, 
operate, maintain, and logistically 
support systems or equipment. 

The systematic definition, evalua- 
tion, coordination and disposition of each 
proposed change, deviation, or waiver, and 
the implementation of each approved change 
in the configuration of a 
Program/Project/EI after formal 
establishment of the configuration 
identification. 
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GLOSSARY OF TERMS (contd.) 


Configuration Control Board: The functional body within a Program/ 

Project/EI office responsible for 
directing the implementation of baselines, 
and the review and disposition of all 
changes, deviations, or waivers to these 
baselines. 

Configuration Inspection: The last of the formal baseline reviews in 

which the manager conducts an audit of his 
CEI to determine that the "as-built" 
documentation is in accordance with its 
release engineering, to approve the CEI 
specification and to establish the product 
configuration baseline. 

Configuration Management: A discipline applying technical administra- 

tive direction and surveillance to: 


Identify and document the technical 
requirements of a Program/Project/EI. 

Control changes/deviations/waivers to 
these technical requirements. 

Record and report change processing and 
implementation status. 

Construction Analysis: An internal inspection for assessing 

design, workmanship, assembly, and any 
other processing associated with 
fabrication of the part. 

Contractor: Any person, partnership, company, or 

corporation (or any combination of these) 
which is a party to a contract with the 
United States Government. 


Countdown Demonstration 

Test - Dry: A dress rehearsal, with flight crew 

participation, of the Lauch Countdown 
operations excluding external tank 
propellant loading opertions. 
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Countdown Demonstration 
Test - Wet: 


GLOSSARY OP TERMS (contd.) 


A full dress rehersal of the launch 
countdown operations including cryogenic 
propellant loading of the external tank. 
Test normally terminates at the time for 
Space Shuttle Main Engine (SSME) 

Ignition. Flight crew does not 
participate from the vehicle. 

CR: Configuration Review 

Critical Design Review: A formal technical evaluation of the 

detailed design of an end item or series 
of end items to assure that the detailed 
design meets the specified requirements 
for the end item or end items. This 
evaluation may be conducted in one review 
or a series of phased reviews as 
appropriate for the complexity of the end 
item(s) . 


Critical Hazard: 

CRR: 

PAR: 

DE: 


A hazard with the potential for resulting 
in damage to equipment or for requiring 
use of contingency or emergency procedures. 

Cargo Readiness Review 

Deviation Approval Request 

Design Engineering 


DCR: Design Certification Review 

DDT&E : Design, Development, Test and Evaluation 

Design Certification Review: A formal comprehensive evaluation of all 

hardware and software end items and 
elements of a system to assure and certify 
that the overall system and the system sub 
elements meet the design requirements and 
are satisfactory for accomplishment of a 
safe and successful mission. 


Design criteria: 


Standards upon which a design is based. 


Design Review: A systematic, technically oriented and 

documented evaluation of spacecraft, 
experiments, end items and unique support 
equipment by a team of specialists. 
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GLOSSARY OF TERMS (contd.) 


Design Review Program: 


Design Specification: 


Designated Representative: 


Detail Drawing: 


Deviation: 


A systematized and disciplined application 
of the broad technical competence of the 
contractor and the customer to a product. 
The program is intended to improve the 
product and to provide assurance to 
contractor and customer management, by 
formalized documentation of the decision 
logic, that the most satisfactory design 
has been selected to meet the program 
requirements. 

A document prescribing criteria to be 
satisfied in designing a particular 
component, subsystem, or system (or 
part). Typical criteria include 
performance requirements under specified 
environments, interface requirements, 
size, weight, ruggedness, safety margins, 
derating factors, and apportioned 
reliability goal (with definition of 
failure) 

An individual (such as a NASA plant repre- 
sentative) , firm (such as an assessment 
contractor), or Government agency 
designated and authorized by NASA to 
perform a specific function(s) relative to 
the contractor's reliability effort; e.g., 
monitor-ship, assessment, and design 
review participation and/or approval of 
certain documents or actions. 

Delineates information to describe an item 
and shall include materials, finish, 
tolerances, and other requirements as 
applicable. 

A specific authorization, granted before 
the fact, to depart from a particular 
requirement of specifications or related 
documents. 


DR: Disposition Record 

Drawing Departure 

Authorization : A form completed by the contractor and 

submitted for each noncompliance condition 
which requires a departure from the design 
requirements set forth in the contractual 
Interface Control Document 
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DRT: 

DWG: 


EC: 


E&D : 

El: 

Emergency: 


End Item: 


GLOSSARY OF TERMS (contd.) 

Design Review Team 
Drawing 

Engineering Change 
Engineering Development 
End Item 

Any condition that can result in crew 
injury or threat to life and that requires 
immediate corrective action, including 
predetermined crew response. 

A space system or any of its principal 
system or subsystem elements; for example, 
launch vehicle, spacecraft, ground support 
system, propulsion engine, CEI or guidance 
system. Also, articles covered by major 
subcontracts where NPC 200-2 is invoiced by 
the NASA installation or by a system prime 
contractor. Also, articles which will be 
delivered directly to a Government 
Installation or provided as GFP to a 
contractor. 


Engineering Change: An authorized modification to a baseline. 

Engineering Change Proposal: The method by which a contractor submits a 

proposed change to a configuration 
baseline for complete assessment by the 
procuring activity. 

Engineering Drawing: A verifiable engineering delineation 

setting forth pictorial or descriptive 
language representations or combinations 
thereof. Drawings solely for contractor's 
processes or plant facilities are not 
included. 

A separate item designed either to provide 
an individual service or function as a 
self-contained unit, or to perform a 
service or function when used in 
conjunction with other devices. 

Eastern Space and Missile Center 


Egulpment : 


ESMC: 


ET: 


External Tank 



GLOSSARY OF TERMS (contd.) 


ETR: 

Experiment: 


FAL: 

Facility: 


FCB: 

FCF: 

Feasibility Study: 


FEC: 

Field Engineering Change: 
Flight Readiness Firing: 


Flight Readiness Review: 


FMCF: 


FMEA: 


Eastern Test Range 

An action or a process undertaken to 
confirm or disprove an existing theory and 
to obtain data leading to the development 
of new theories. 

First Approach and Landing 

Any fixed installation; e.g., test stand 
or launch platform, which is part of a 
program/project. This includes Real 
Property Installed Equipment (RPIE). 

Flight Certification Board 

First Captive Flight 

The phase during which studies are made of 
a proposed item or technique to determine 
the degree to which it is practicable, 
adviseable, and adaptable for the intended 
purpose. 

Field Engineering Change 

The method for expediting make fit/make 
operable changes at NASA Using Sites. 

A 20-second firing of the Space Shuttle 
Main Engines (SSMEs) in a near Shuttle 
flight configuration on the launch pad, 
with a scheduled cutoff prior to Solid 
Rocket Booster (SRB) ignition and liftoff, 
to be conducted (one-time-only) as part of 
the CDDT. 

The last major evaluation prior to a launch 
conducted as a consolidated review of the 
hardware, software, operational and 
support elements to assure readiness to 
begin the mission. The result of this 
evaluation, shortly before launch, is a 
positive certification of flight readiness 
by both government and contractor. 

First Manned Captive Flight 

Failure Modes and Effects Analysis 
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GLOSSARY OF TERMS (contd.) 


FRF: 

FRR: 

GFE: 

GFP: 

GMI: 

GOR: 


Flight Readiness Firing 
Flight Readiness Review 
Government Furnished Equipment 
Government Furnished Property 
Goddard Management Instruction 
Ground Operations Review 


Ground Support Equipment : Nonflight equipment, implements, and 

devices required for the handling, 
servicing, inspection, testing, 
maintenance, alignment, adjustment, 
checking, repairing, and overhauling of an 
operational end time or a subsystem or 
component part thereof. This may include 
equipment required to support another item 
of GSE. 

Ground Support Equipment 

Goddard Space Flight Center 

Health and Safety Engineering Office 

The presence of a potential-risk situation 
caused by an unsafe act or condition. 

Interface Control Drawing 

Installation Drawing: Shows general configurations, attaching 

hardware, and information to locate, 
i position, and mount an item relative to 

fixed points and to other items. 

Interface: The point or area where a relationship 

exists between two or more parts, systems, 
programs, persons, or procedures wherein 
physical and functional compatibility is 
required. 

Interface drawing: The engineering drawing which graphically 

or descriptively displays the conditions 
of the interface which exist between 
assemblies. 


GSE: 

GSFC: 

H&SEO: 

Hazard: 

ICD: 
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GLOSSARY OF TERMS (contd.) 


Integration: 


JG: 

JPL: 

JSC: 

KMI: 

KPD: 

KSC: 

LaRC: 

LeRC: 

Launch Integrity Team: 


Launch Vehicle: 


LIT: 

LHB: 

LMI: 

L-2 Day: 
LO: 


The process of assuring that the major 
elements of a program be conceived, 
designed, assembled, tested, operated, and 
documented In such a manner as to be 
compatible with each other and to satisfy 
the program objectives. 

JSC Ground Systems 

Jet Propulsion Laboratory 

Johnson Space Center 

Kennedy Management Instruction 

Kennedy Program Directive 

Kennedy Space Center 

Langley Research Center 

Lewis Research Center 

Review team composed of high-level 
representatives of the prime contractor, 
or the integration contractor and the 
major associates, to review each Launch 
Vehicle prior to the customer 
flight-readiness review. 

The part of the space vehicle which 
furnishes the propulsion and guidance 
during the inital part of the trajectory 
to provide the prescribed velocity, 
position, and attitude required for 
injection into the desired trajectory. 
Launch vehicles are commonly called 
boosters and consist of two or more 
propulsive stages. 

Launch Integrity Team. 

Langley Handbook 

Langley Management Instruction 

Launch Minus 2 Days 


Launch Operations 



GLOSSARY OF TERMS (contd.) 


LRR: 


Launch Readiness Review 


LRU: 

LVP: 

Man-Rated Space Vehicle: 


MATCO: 

MCR: 

Milestone: 


Mission: 


Mission Task: 


MMM: 

HMI: 

Modification: 

Modification Instructions: 


MR: 


MRB: 


MSEC: 


Line Replaceable Unit 

Launch Vehicle Project 

Space vehicles for manned flight which 
have achieved the standards of performance 
and reliability previously established as 
reasonably acceptable for this class of 
equipment . 

Material Analysis Tracking and Control 

Master Change Record 

Any significant event in the design and 
development of a space system or in the 
associated program or project which is 
used as a control point for measurement of 
progress and effectiveness or for planning 
or redirecting future effort. 

The performance of a coherent set of 
investigations or operations to achieve 
program goals. 

The specified purpose for which a device 
must perform. 

Marshall Management Manual 

Marshall Management Instruction 

An identified, approved and contractually 
authorized change to systems/equipment and 
spares already accepted by the Government. 

A form initiated by the contractor to be 
used as a checklist for modification kit 
completeness and to serve as instruction 
for accomplishment of the modification. 

Material review disposition 

Material Review Board 

Marshall Space Flight Center 
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GLOSSARY OF TERMS (contd.) 


NASA: 

National Aeronautics and 
Space Administration 


NDE: 

NHB: 

NMI: 

Nonstandard Part: 

Normal STS Mission Phases: 


NSS: 

NSSDC: 

O&C: 

O&H: 

OFT: 

OHA: 

OMD: 

OMI: 

Operational: 
Operational Phase: 


National Aeronautics and Space 
Administration 


Civilian agency with research and 
development jurisdiction in aeronautical 
and space activities, except those 
activities peculiar to and primarily 
associated with the development of weapons 
systems and military operations. 

Non-Destructive Evaluation 

NASA Handbook 

NASA Management Instruction 

One for which no published standard or 
specification exists 

All portions of the mission to be 
performed by the STS, excluding STS abort 
and emergency landing. 

NASA Safety Standard 

National Space Science Data Center 

Operations and Checkout 

Operations and Maintenance 

Orbital Flight Test 

Operations Hazards Analysis 

Operation and Maintenance Documentation 

Operation and Maintenance Instruction 

Equipment for which all research and 
development has been completed with 
achievement of performance objectives. 

The period from acceptance by the user of 
the first operating unit until disposition 
of the system equipment. The operational 
phase overlaps the acquisition phase. 
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GLOSSARY OP TERMS (contd.) 


Operations and Maintenance 
Instruction: 


OPO: 

OPR: 

OSTS: 

OV: 

PAD: 

Part: 


PASS: 

Payload: 


Payload Bay: 


Payload Elements: 

PDP: 

PDR: 

PERT: 


Organized step-by-step procedural 
information specifying methods of 
operating and maintaining flight hardware 
and support equipment. OMIs are used in 
the performance of day-to-day operations 
and maintenance tasks. 

Orbiter Project Office 

Office of Planning and Review 

Office of Space Transportation Systems 

Orbiter Vehicle 

Project Approval Document 

One piece or two or more pieces joined 
together which are not normally subject to 
disassembly without destruction. 

Planning and Scheduling System 

A specific complement of instruments, 
space equipment, and support hardware and 
software carried into space by one shuttle 
flight to achieve specific objectives (not 
considered part of the basic STS itself). 

The 15-foot diameter by 60-foot long 
enclosed volume within the orbiter, 
designed to carry carriers, payloads, 
payload-support equipment, and associated 
mounting hardware. 

Instruments, support equipment, software, 
or other individual payload items that are 
subsets of an integrated cargo complement. 

Program Development Plan 

Preliminary Design Review 

Program Evaluation and Review Technique 
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GLOSSARY OF TERMS (contd.) 


Piggyback Experiment: 


PIP: 


PM: 


An experiment which rides along with the 
primary experiment on a space-available 
basis without interfering with the mission 
of the primary experiment. 

Payload Integration Plan 

Program Management 


Preliminary Design Reviews: A formal technical evaluation of the 

proposed design approach to an end item. 

It is conducted prior to the detailed 
design or very early in the detailed 
design phase to asure that the engineering 
approach is acceptable. As a general rule 
this evaluation is conducted when the 
design layouts are 95% completed and the 
detailed design is 10% completed. 


Program: A related series of undertakings designed 

to accomplish a broad scientific or 
technical goal. Attainment of such 
long-range goals may be accomplished by 
implementation of specific projects. 


Program Evaluation and 

Review Technigue: Method of charting events and obtaining 

predicted performance in accordance with a 
schedule. 


Program/Prolect 

Reguirements Review: The first of the major baseline reviews, 

in which the Program/Project Manager 
assures that the mission objectives and 
technical requirements of his 
specification are fully understood and can 
be met based on the concepts selected for 
each El/major element and that the 
Program/Project Requirements Baseline is 
established. 

Project: A scheduled undertaking, within a program, 

which may involve the research and 
development, design, construction, and 
operation of system and associated 
hardware, or hardware only, to accomplish 
a scientific or technical objective. 
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GLOSSARY OF TERMS (contd.) 


PRR: 

PSO: 

QA: 

QAI: 

Qualification Tests: 


R&QA: 

Released Engineering: 


Retrofit: 

RID: 

RPIE: 

RR: 

SAA: 
Safety : 


Preliminary Requirements Review 

Project Safety Officer 

Quality Assurance 

Quality Assurance Instructions 

Tests intended to demonstrate that the 
design will function within performance 
specifications under simulated conditions 
more severe than those expected from 
ground handling, launch, and orbital 
operations. Their purpose is to uncover 
deficiencies in design and method of 
manufacture. They are not intended to 
exceed design safety margins or to 
introduce unrealistic modes of failure. 

Reliability and Quality Assurance 

The current and total set of approved 
drawings and specifications which have 
been completed for a product and formally 
recorded. It is centrally controlled and, 
upon release, represents the engineering 
requirement to which all contractor 
disciplines applicable to that product 
must adhere in order to meet contract 
requirements. 

The incorporation of an engineering change 
in accepted or in-service items. 

Review Item Disposition 

Real Property Installed Equipment 

Requirements Review 

Systems Assurance Analysis 

Freedom from chance of personnel injury or 
fatality and damage to or loss of 
equipment or property. 
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GLOSSARY OF TERMS (contd.) 


A process used in lieu of, or in addition 
to, testing to validate compliance of an 
item to specification requirements. The 
techniques include the use of analytical 
models in systems engineering analyses, 
safety analyses, statistics and 
qualitative analyses, computer and 
hardware simulations, and analog modeling. 

Systems or subsystems whose inadvertent 
operation or failure to operate results in 
either a critical or catastrophic hazard. 

Schematic Diagram: A diagrammatic drawing that shows function 

symbols with interconnections to 
illustrate circuit operation. It does not 
necessarily identify physical location of 
components or connections between them. 

SFP: Single Failure Point 

Single Failure Point: A single element of hardware, where 

failure would lead directly to loss of 
life, vehicle, or mission. When safety 
considerations dictate that an abort be 
initiated when a redundant element fails, 
the element is also considered a SFP. 

The orbiter, solid-rocket boosters, and 
external tank. 

A system of equipment consisting of launch 
vehicle(s), spacecraft, ground support 
equipment, and test hardware, used in 
ground testing, launching, operating, and 
maintaining space vehicles or spacecraft. 

Space Transportation System: An integrated system consisting of the 

Space Shuttle (Orbiter, External Tank, 
Solid Rocket Booster, and Flight Kits), 
Upper Stages, Spacelab, and any associated 
flight/ground hardware and software. 

Spacecraft : Systems, manned or unmanned, that are 

designed to be placed into a orbital 
trajectory, orbit about the Earth, or a 
trajectory to another celestial body. 


Space Shuttle: 
Space System: 


Safety Analysis: 


Safety Critical: 
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GLOSSARY OF TERIS (contd.) 


SP&R: 

SR&OA: 

SRB: 

SSME: 

STS: 

STS Operator: 


System: 


TRR: 

TSPD: 

UA: 

Validation: 


VCR: 

Verification: 


Safety Policy and Requirements 

Safety, Reliability and Quality Assurance 

Solid Rocket Booster 

Space Shuttle Hain Engines 

Space Transportation System 

The NASA Headquarters/STS Operations 
Directorate is the STSoperator. NASA/ JSC 
is the STS flight operator and is 
responsible for the safety of payload 
flight systems and flight operations. 

One of the principal functioning entities 
comprising the project hardware and 
related operational services within a 
project or flight mission. Ordinarily, a 
system is the first major subdivision of 
project work. Similarly, a subsystem is a 
major functioning entity within a system. 
(A system may also be an organized and 
disciplined approach to accomplish a task; 
e.g., a failure-reporting system.) 

Test Readiness Review 

Technology & Space Program Development 

Unexplained Anomoly 

The inspection and tests necessary to 
establish confidence for a system added or 
to restore confidence in a system 
invalidated by the installation of a 
modification into a Contract End Item. 

Vehicle Configuration Review 

Determination by analysis, inspection, 
test or a combination of- the three that a 
product can function as intended in a 
particular mission; this includes 
assurance that the product design has been 
qualified and that each product item has 
met acceptance criteria. 
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GLOSSARY OF TERMS (contd.) 


Verification 


Waiver : 


WBS: 


Procedure: A document that implements one or more 

specific activities required by the 
verification plan. The verification 
procedure describes details of each 
activity such as instrumentation, data 
collection, and reporting. It includes 
provisions for safety and contamination 
control that are associated with the test 
being conducted. 

A written authorization accepting a 
departure, after occurrence, from a 
baseline requirement, normally limited to 
a single application of GEI. 

Work Breakdown Structure 
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APPENDIX B 


NASA DOCUMENTS REVIEWED 


This appendix contains a list of the supplied NASA documents reviewed 
in this study. It also contains a list of the FAA and DoD documents 
reviewed. 
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AMES RESEARCH CENTER 
DOCUMENTS 


NUMBER 
AHB 1700-1 


AHB 1710-3 

AHB 1740-2 
AMM 1740-1 

AMM 1760-1 

AMM 1770-1 


TITLE AND DATE 

AMES Health & Safety Manual 

Chapter 5: Facility Operational Safety 

Chapter 9: Aviation Safety 

Chapter 10: Pressure System Safety 

No Date 

Certification and Recertification of 
Ground Based Pressure Vessels and 
Pressurized Systems - June 1981 

Outline for Airworthiness and Flight 
Safety Reviews - January 1975 

Management Manual 

Airworthiness and Flight Safety Reviews 
February 18, 1975 

Management Manual 
Man-Rating of Simulators 
September 19 , 1980 

Management Manual 

Project Safety Reviews of Major 

(Discrete) Construction of Facility 

Modification 

September 19, 1980 
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GODDARD SPACE FLIGHT CENTER 
DOCUMENTS 


NUMBER 
GMI 1700.3 


GMI 5330.7 


GMI 80 10. IB 


TITLE AND DATE 


Management Instruction 

System Safety for Flight Projects 

September 8, 1981 

Management Instruction 
Development and Implementation of 
Environmental Verification Requirements 
for Space Flight Hardware 
December 9, 1979 

Management Instruction 
Spacecraft Design Review Program 
March 22. 197 
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JET PROPULSION LABORATORY 
DOCUMENTS 


NUMBER 

JPL D-363 
OER 82-4 

1-28 

MJS77-RLH-76-09 

PD618-210 


PD6 18-223 


JPL-750-150 


TITLE AND DATE 

Guidelines for Planning and Conduct of 
Formal and Informal Reviews 
February 7, 1983 

Policy 

Work Output and Proposal Review Policy 
August 28, 1978 

Hardware Reviews - April 19, 1976 

Project Document 
Mariner Jupiter/Saturn 1977 
Spacecraft Procurement Instructions 
January 10, 1974 

Project Document 
Mariner Jupiter/Saturn 1977 
Spacecraft System Review Program 
April 15, 1975 

Flight Experiment Requirements and 

Guidelines 

July 1981 
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JOHNSON SPACE CENTER 
DOCUMENTS 

NUMBER 

TITLE AND DATE 

LA2 

Space Shuttle Program 
System Integration 
Flight Readiness Review Planning 
October 8. 1980 

LA3-82-059 

Flight Readiness Review (FRR) and 
L-2 Day Review 
September 23, 1982 

LC-82-224 

Flight Readiness Review (FRR) and 
L-2 Day Review Revised 
September 23, 1982 

JSC07700 

Configuration Verification, Volume IV 
No Date 

None 

Space Shuttle Program 

Orbital Flight Test Design Certification 
Review Plan 
December 21, 1978 

None 

Space Shuttle Flight Readiness Review Plan 
February 26, 1982 

None 

Space Shuttle Vehicle 
Configuration Review (VCR) Plan 
March 15, 1982 

JSC- 16898 

Space Shuttle Orbiter Flight Readiness 
Review Plan, Orbital Flight Test Plan 
Revision A 
June 1981 

JSC-17453 

Space Shuttle Orbiter Configuration 
Verification Review Plan, Operational 
Orbiter Vehicles 
September 1981 

JSC-14046 

Payload Verification Requirements 
March 1982 

None 

Space Shuttle Orbiter Review Plan 
Preliminary Design Review 
Critical Design Review 
January 11, 1974 
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JOHNSON SPACE CENTER (contd.) 


NUMBER 
JSC 16898 

JSC 08117 
Rev. B 

None 

JSC 11758 
JSC 13830 


TITLE AND DATE 

Space Shuttle Orbitec Flight Readiness 
Review Plan, Orbital Flight Test STS-1 
September 1980 

Space Shuttle Program 

Procedure Cor Certification of Flight 

Readiness 

Hay 17, 1978 

Space Shuttle Program 
System Integration, Flight Readiness 
Review Planning 
October 8, 1980 

Orbiter Flight Readiness Review Plan for 
Approach and Landing Test 
November 1976 

Implementation Procedure for STS 
Payloads, System Safety Requirements 
May 1979 
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KENNEDY SPACE CENTER 
DOCUMENTS 


NUMBER 
DE-ID8020 . 1A 


K-SM-04 .8.3 


KPD 8620.5 


TITLE AND DATE 
Implementing Directive 

Design Engineering Shuttle Transportation 
System Readiness Review Cycle 
September 15, 1981 

STS- 3 and Subs 

Readiness Review Management Plan 
March 1982 

Program Directive 
Space Transportation System Flight 
Readiness Reviews 
August 7 , 1979 
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LANGLEY RESEARCH CENTER 
DOCUMENTS 


NUMBER TITLE AND DATE 

1710.1 Management Manual 

Human Factors Research 
Man-Rating Requirement, and Committee 
Review Procedures 
June 22, 1977 

LHB 7121.1 Announcement 

Langley Proposals for New Starts 
May 9, 1977 

LHB 7121 Project Management, Handbook 

December 1975 

LMI 7000.2 Management Manual 

Review Program for Major Construction, 
Rehabilitation, and Modification Projects 
of Research Facilities 
March 26, 1981 

LMI 7120.1 Management Manual 

Aeronautic and Space Flight Projects and 
Experiments Review Program 
October 21 , 1980 
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LEWIS RESEARCH CENTER 
DOCUMENTS 

NUMBER 

TITLE AND DATE 

01-14 

Rev. 4 

Operating Instruction 

ATLAS/CENTAUR Launch Vehicle Hardware and 
Software Review Milestones 
July 1. 1980 

01-19 

Rev. 2 

Operating Instruction - 19 
LVD Procedure for Review and Disposition 
of Launch Vehicle System Quality Problems 
July 1. 1980 

01-20 

Rev. 2 

Operating Instruction - 20 

Launch Vehicle Division Critical Problem 

List 

July 1. 1980 

01-22 

Rev. 2 

Operating Instruction - 22 
System Engineer's Duties and 
Responsibilities 
July 1, 1980 

01-23 

Rev. 3 

Operating Instruction -.23 

Launch Vehicles Division Engineering 

Review Board (ERB) 

July 1. 1980 

01-24 

Rev. 3 

Operating Instruction - 24 
Processing of Changes to the Centaur 
Unified Test Plan (UTP) 

July 1980 

01-28 

Rev. 1 

Operating Instruction - 28 
Processing of Software Change Sheets 
July 1. 1980 

01-30 

Rev. 1 

Operating Instruction - 30 
Unscheduled Removal of Atlas/Centaur 
Flight Hardware after Vehicle Erection 

None 


Aircraft Operations Manual 

Appendix B, Aircraft Operations Safety 

Assurance Plan - No Date 

Appendix C, Flight Workorder Control 

System - No Date 
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MARSHALL SPACE FLIGHT CENTER 
DOCUMENTS 


NUMBER 
MM 2314.2 

MMI 8010.5 
MMI 8030. 2A 
MM 8040. 12A 

MMI 8040. 15B 


TITLE AND DATE 

Data Management Manual 
KSFC Data Management 
Operating Procedures 
March 15, 1972 

Management Instruction 
MSFC Baseline Design Review 
July 7, 1980 

Management Instruction 
Policy on MSFC Payloads 
March 4, 1980 

Standard Contractor 
Configuration Management Requirements 
MSFC Programs 
March 3, 1981 

Management Instruction 
Configuration Management 
May 12, 1976 
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NUMBER 

E-885-83-01 

SP-6502 

ESA Contract N° 
4846/81/NL/PP(SC) 

NMI7121.1C 

None 

None 
1700. 7A 

403-174-001 

None 

None 

None 

None 

None 

None 


NASA HEADQUARTERS 
DOCUMENTS 


TITLE AND DATE 

Prelaunch, Mission Operation Report 
Infrared Astronomical Satellite (IRAS) 
Launch - January 3, 1983 

Elements of Design Review for Space 
Systems 

A Comprehensive Study on Project Review 
Techniques Final Report - December 31, 

1982 

Management Instruction - Planning and 
Approval of Major Research and 
Development Projects - March 24, 1977 

ST-S-5 EMU Interim Report to the 
Independent Anomaly Review Team - 
December 17, 1982 

Certification Process: Lithium Batteries 

December 17, 1982 

Safety Policy and Requirements for 
Payloads Using the Space Transportation 
System (STS) - December 9, 1980 

Safety Reviews - No Date 

Certification Review Process January 3, 

1983 

Design Certification Review of New 
Designs for STS- 5 and STS-6 - December 
22, 1982 

Outline - STS Payload Project Review 
Guidelines - June 10, 1979 

An Independent Review of the Flight 
Certification Program - No Date 

Space Transportation Systems (STS) 
Certification Review - November 13, 1979 

Charter - Special Staff to the Chief 
Engineer - No Date 
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NASA HEADQUARTERS (contd.) 


NUMBER 
NMI 8010.1 


NHB5300.4(1B) 


NHB5300 .4(1A) 


TITLE AND DATE 

Classification of NASA Space 
Transportation System (STS) Payloads - 
September 26, 1979 

Reliability and Quality Assurance 
Publication Quality Program Provisions 
for Aeronautical and Space System 
Contractors - April 1969 

Reliability and Quality Assurance 
Publication Reliability Program 
Provisions for Aeronautical and Space 
System Contractors - April 1970 
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DoD DOCUMENTS REVIEWED 


NUMBER 

AR70-37 

NAVMAT INST 4130 
MCO 4130. 1A 
APR 65-3 
DSAR 8250.4 
NAS/CSS 80-14 
DNA INST 5010.18 

AFR 800-2 


APR 57-1 


MIL-STD-480A 


MIL-STD-48 1 A 


TITLE AND DATE 

Configuration Management 
1A July 1. 1974 


Acquisition Program Management 
August 13, 1982 

Statement of Operational Need (SON) 

June 12, 1979 

Configuration Control-Engineering Changes 
Deviations and Waivers 1 
December 29 , 1978 

Configuration Control-Engineering Changes 
Deviations and Waivers (Short Form) 
October 18, 1972 
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FRA DOCUMENTS 


NUMBER TITLE AND DATE 

8110.4 Type Certification 

December 27 , 1967 
Reprint October 1978 
Includes Changes 1 through 22 

FAA Aviation Regulations, Change 3, 

Part 25 - Airworthiness Standards: 
Transport Category Airplanes 
February 10, 1977 

Southern Region FAA Flight Standards 
Division Application Guide for Obtaining 
Supplemental Type Certificate 
March 5, 1965 

8110.21 Supplemental (Type) Certificate (STC) 

Approvals, "One Aircraft Only' 1 
September 22, 1976 

8110. 10C FAA Approvals of Major Modifications/ 

Alterations 

January 30, 1978 
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APPENDIX C 


NASA DOCUMENTS REFERENCED 


This appendix contains a list of the NASA documents referenced in one 
or more of the documents supplied for review. 
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NASA DOCUMENTS REFERENCED 


NMI 7121.1, "Planning and Approval of Major Research and 
Development Projects" 

NHB 7121.4, "Guidelines for Project Planning" 

LMI 1710.1, "Human Factors Research, Man-Rating Requirements, 
and Committee Review Procedures" 

LHB 7121.1, "Project Management" 

MM 8040.12 "Standard Contractor Configuration Management 
Requirements" 

MMI 8030.2 "Policy on MSFC Payloads" 

MMI 8040.15 “Configuration Mangement" 

MM 2314.2 "fBFC Data Management Operation Procedures" 

KMI 2530.4, "Transcription, Monitoring, Interception, or 
Recording or Telephone or other Conversations, or the 
Proceedings of Meetings" 

KMI 5310. 9B. "Safety Hazard and Reliability Analyses of Ground 
Support Equipment and Facilities, and Launch Operations 
Operational and Integrated Hazards Analyses" 

OSTS "Space Shuttle Flight Test Operations Directive," 

Revision C. dated January 1979 

KPD 8010. 2A. "Certificate of Flight Readiness Sign-off for KSC" 

KSC KPD 8620.5, STS Flight Readiness Reviews 

K-SM-04.8.2, STS-2 Readiness Review Management Plan 

K-SM-04.8.1, STS-1 Readiness Review Management Plan 

GMI 8010. IB Design Review Program, March 22, 1973 

NHB 5300. 4 (3A) Requirements for Soldered Electrical Connections 

NHB 5300. 4 (IB) Quality Program Provisions for Aeronautical and 
Space System Contractors 
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NASA DOCUMENTS REFERENCED (contd.) 


NHB 5300. 4(1A); Reliability Program Provisions for Aeronautical 
April 1970 and Space Systems Contractors (Applies only as 

defined in 612-16) 

NHB 5300.4(10 Inspection System Provisions for Suppliers of 
Space Materials, Parts, Components and Service 

JSC-14046 "Payload Verification Requirements" March 1982 

NHB 1700. 7A "Safety Policy and Requirements - For Payloads Using 
the STS'* December 9, 1980 

JSC-07700, "Configuration Management Requirements" Jan. 31, 1981 
Vol. IV 

"Space Shuttle Program Orbital Flight Test Design 
Certification Review Plan" December 21, 1978 

"Space Shuttle Flight Readiness Review Plan" 

February 15, 1980 

WC2/80-308 "JSC Document No. 16898, Space Shuttle Flight 
Letter Readiness Review Plan for Orbital Flight Test, 

STS-1" September 29, 1980 

JSC-16898 "Space Shuttle Orbiter Flight Readiness Review Plan" 
September 1980 

"Space Shuttle Program System Integration Flight 
Readiness Review Planning" October 8, 1980 

"Space Shuttle Flight Readiness Review Plan" 

February 26, 1982 

"Space Shuttle Vehicle Configuration Review (VCR) Plan 
March 15, 1982 

LC-82-224 "Flight Readiness Review (FRR) and L-2 Day Review" 
Letter September 23, 1982 

"Space Shuttle Flight Readiness Review Plan" 

September 1, 1982 

LA3-82-059 "Flight Readiness Review (FRR) and L-2" 

September 23, 1982 


C-3 



NASA DOCUMENTS REFERENCED (contd.) 


, NMI 1052. 12A, "NASA-DOD Agreement for Performance of Contract 
Administration and Contract Audit Services in Support of NASA 
Contracts" 

. NHB 1200.1, "Basic Management Processes" 

. NHB 1700. l(vi), "Basic Safety Requirements" 

. NPD 2220.3, "Relationship Between Scientific Investigations Under 
Contract to NASA and NASA Contract Monitors" 

. NHB 2410. 1A, Management Procedures for ADP Equipment" 

. 2570.3, "Radio Frequency Management Manual" 

. 5100.2, "NASA Procurement Regulations" 

. 5101. 1A, "Requirements for Legal Review of Procurement Matters" 

. 5101.19, “Avoiding Conflict-of-interest Situations in Placing of 

NASA Contracts" 

. NMI 5103. 4B, "Source Selections by the Administrator in 
Competitive Procurements" 

. NMI 5104.1, "NASA Policy and Procedures for Use of Contracts for 
Nonpersonal Services" 

. NHB 5104. 3A, "DOD and NASA Incentive Contracting Guide" 

. 5104.4, "Cost Plus Award Fee Contracting Guide" 

. NPD 5300. 7A, "Basic Policy on Safety and Reliability and Quality 
Assurance" 

. NHB 5300. 4(1A), "Reliability Program Provisions for Aeronautical 
and Space System Contractors 

. NHB 5600.2, "Statements of Work Handbook" 

. NHB 5610.1, "Handbook for Preparation of Work Breakdown 
Structures" 

. NPD 7000. 1A, "Authorization and Control of Agency Programs and 
Allocation of Resources" 

. NHB 7100. 5B, "Launch Vehicle Estimating Factors for Advance 
Mission Planning" 
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NASA DOCUMENTS REFERENCED (contd.) 


. NMI 7100.11, "Announcement of Opportunity Process - Acquisition 
and Administration of Space Science Investigations" 

. NMD/A 7110. 1A, "To Approve Advanced Studies of Possible 
Aeronautical and Space Missions" 

. NMI 7121. IB, "Planning and Approval of Major Research and 
Development Projects" 

. NHB 7121.2, "Phased Project Planning Guidelines" 

. NHB 8030. 4G, "Space Flight Payload Capability" 

. NMI 8030.5, "Submission and Selection of Advanced Research and 
Technology Space Flight Experiments" 

. NHB 8030.6, "Guidelines for Acquisition of Investigations" 

. NPD 8621. 1A, Investigation of Accidents and Mission Failures" 

. NHB 9090.6, "Guidelines for Evaluation of Contractor Accounting 
Systems" 

. 9501. 2A, “Procedures for Contractor Reporting of Correlated Cost 

and Performance Data" 

. LHB 1700.1, "Safety Program" 

. LHB 1700.6, "Control of Hazardous Materials" 

. LHB 2310.1, "Resource Management System" 

. LMI 5000.2, "Procurements" 

. LHB 5000.2, "Basic procurement Guide" 

. LMI 5300.1, " LRC Reliability and Quality Assurance Management" 

. LMI 700.2, "Reviews of Major Construction or Facility 
Modification Projects" 

. LHB 7100.1, "Research and Development Program Management 
Procedures" 

. LMI 7100.3, "Procedures for the Use of Purchase Requests in 
Research and Development Procurements" 

. .MI 7120.1, "Review of Small Flight Projects and Experiments" 
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NASA DOCUMENTS REFERENCED (contd.) 


LMI 7120.2, "Authority and Respnslbilities of Managers of Small 
Space Flight Projects" 

LMI 9100.1, "Job Orders" 

LMI 9500.1, "Contractor Financial Management Reporting" 

SP-6501, "An Introduction to the Evaluatio of Reliability Program" 

SP-6502, "Elements of Design Review for Space Systems" 

SP-6505, "Parts and Materials Application Review for Space 
Systems" 
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APPENDIX D 


REVIEW DESCRIPTIONS 


This appendix contains a description of different reviews shown in 
selected documents supplied for the seven (7) centers. The centers are 
not called out by name since it was not the purpose of this review to 
audit the current documentation of individual centers. The method of 
presentation varies since the information shown is extracted directly from 
the documents supplied. The general format of the review descriptions 
follows: 


- Procedures : procedures followed in organizing and conducting 

reviews . 

- Personnel : personnel making up the Review Board. 

- Responsibilities : responsibilities of the Review Board personnel. 

- Review Items/Data : items and data to be reviewed by the review 

board for the specific review as called for in the review agenda. 

- Feedback : description of the method for informing the review 
board of actions taken on all action items. 

- Closeout : description of the method described for closing-out 

actions items. 
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CENTER NO. 1 


REVIEWS 

DOCUMENT NO. 1-1 
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CENTER NO. 1 
DOCUMENT NO. 1-1 

PROCEDURES 


Formal Reviews are conducted when the work or p. oposal exhibits one 
or more of the following characteristics: 

The activity represents a large dollar volume in a given fiscal 
year or in total 

- The products are highly visible deliverables 

Success or failure to produce will impact important future 
assignments 

The conduct of the activity involves matters of substantial 
institutional concern 

- The activity reports directly to an Assistant Laboratory 
Director 

The activity is one for which an Assistant Laboratory Director 
or Deputy Laboratory Director is the convening authority 

Criteria for the large dollar volume are arbitrarily recommended as 
being expenditures in excess of $1,000,000 in a given fiscal year or 
$3,000,000 in total resources for all fiscal years. The judgment as 
to whether or not tasks require formal reviews is left to the 
responsible organization. 


Standing Review Boards are selected for projects or tasks which have 
a high level of activity and resource requirements. Selection of 
board members by the convening authority is generally made from 
senior JPL technical and management staff. Supplemental technical 
or management supporting members or advisors may be added to the 
board as required by circumstances. If the review board is to 
function over the lifetime of a project it is advisable to select 
extra board members and rotate to cover needs. 

Review Plans. Early in the life of a project or task, or as soon as 
a proposal is initiated, a Review Plan should be prepared which 
identifies all reviews, their schedules, and the make-up of the 
review boards. The Plan should also indicate for each specified 
review the purpose, the convening authority, and the methodology. 
For tasks which will conduct informal reviews, the Proposal Phase 
Checklist ' is recommended. 
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CENTER NO. 1 
DOCUMENT NO. 1-1 

PROCEDURES fcont^d) 


Convening Authority. For major flight projects such as Voyager and 
Galileo, the reviews are classified according to level of importance 
3*(see*' PD ; 625-7)." The convening authority is normally the Program 
^Manager or the Assistant Laboratory Director of the cognizant 
Program Office for level 1 reviews; the Project Manager for level 2, 
and the cognizant System Manager for level 3 reviews. For all other 
reviews the convening authority is the official next in line above 
the manager of the activity being reviewed. The responsibilities 
and functions of the convening authority are given in Section 41. 


Formal Reviews . A written charter is issued by the convening 
authority to identify the scope and objective of the review. This 
charter includes board members, review agenda, location, and a 
timetable for review activities. The charter also includes the 
specific functions the board is to address. 


Egrmal. Reviews* The convening authority names the board chairman 
and, assisted by the board chairman and Project, System, or Task 
Manager, names the review board members. The board must include 
i a majority number of individuals not performing or managing the work 
! being reviewed. The chairman appoints the board secretary. 

Formal Reviews . Background information and review presentation 
material of use to the board members should be distributed to the 
review board members early enough to enable them to examine it prior 
to the meeting. 


Informal Reviews are conducted according to the needs of the project 
or task and they are based on the decision of the convening 
authority. Informal reviews should include the following minimum 
characteristics: 

An assigned chairman 

A review board which includes at least one individual who is 
not managing or performing the week being reviewed 

A presentation of the material being reviewed 

Assigning, tracking, and closing action items 

By this definition meetings related to technical discussions and 
decisions, periodic resource expenditure and project or task status, 
and progress monitoring are not necessarily reviews. 

the convening authority is the official next in line above 
the manager of the activity being reviewed. 
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CENTER NO. 1 
DOCUMENT NO. 1-1 

PROCEDURES fcont'd) 


Informal Review Minimum Lewi. A board chairman and a review beard 
including at least one individual not managing or performing the 
work being reviewed is named by the convening authority. 


Preparation. When technical issues are to be included it is 
sometimes advisable to prereview with technical specialists and make 
results available to the review board. 


Informal Review. Presentations using existing documentation such as 
drawings and reports may be adequate. Prepared material such as 
viewgraphs should be provided to the review board and meeting 
attendees if used. 


Informal Review. Background information and review presentation 
material of use to the board members should be distributed to the 
review board members early enough to enable them to examine it prior 
to the meeting. 



CENTER NO. 1 
DOCUMENT NO. 1-1 
PERSONNEL AND RESPONSIBILITIES 

Convening Authority . The responsibilities of the convening 

authority are to: 

- Determine whether a review will be formal or informal 

- Appoint the review board chairman and review board msnbers 

- Issue the charter describing the scope, authority, and purpose 
of the review 

- Review and accept or reject the review board findings and 
recommendations 

- Report board findings to Deputy Director if appropriate 

- Approve the cognizant manager's response to the review board 
report 

- Approve closure of action items, including responsibility for 
transmital to board secretary 


Cognizant Manager . The responsibilities of the cognizant manager 

(Project, System, or Task) are to: 

- Recommend board chairman and review board candidates to the 
convening authority 

- Contribute to the generation of the review charter 

* Ensure that the review data package is adequate and is 
distributed on schedule 

- Support the review board chairman in developing the review 
agenda 

- Recommend to the convening authority disposition of the 
findings and recommendations of the board 

- Approve and impose the action itens that are generated 

- Prepare for the convening authority's approval the written 
response to the board's report, including how action items will 
be handled 

- Monitor the action item status and report close out of action 
items to convening authority 
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CENTER NO. 1 
DOCUMENT NO. 1-1 

' PERSONNEL AND RESPONSIBILITIES (confd) 

Board Chairman . The responsibilities of the board chairman cure to: 

Establish an agenda, assisted by the cognizant manager, to 
include all information relevant to the purpose of the review 

Specify the date and location of review meetings 

Identify any unique or specific objectives, constraints, or 
goals of review meetings 

- Appoint a secretary 

Chair the review and keep it on subject and on schedule 

Convene the review board as required to deliberate, and arrive 
at findings and recommendations 

Develop, where necessary, a consensus of findings of the board, 
including an assessment of the risk associated with problem 
areas; develop reccranendations for action 

Ensure that a written report of findings, including 
recommendations for actions, is submitted on a timely basis to 
the convening authority with a copy to the cognizant manager 
and the Office of Engineering and Review 

Approve action item closure prior to distribution to board 
members 


Board Secretary . The responsibilities of the secretary are to: 

- Record cerements of the board members 

Assist the chairman in preparing the board report 

Maintain a file of all board actions, including closure 
memoranda 

Maintain a list of action item closures and due dates for 
outstanding actions items 

Board Members. The responsibilities of the review board members are 
to: 


Become familiar with the documentation and presentation 
material and with the objectives and criteria for assessment 
and evaluation prior to the review 

Participate in all scheduled meetings and contribute to the 
fulfillment of the objectives of the review board 

Assist in developing a disposition for the material reviewed, 
make recommendations for action, and contribute to the review 
board report - 
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CENTER NO. 1 
DOCUMENT NO. 1-1 

PERSONNEL AND RESPONSIBILITIES (cont'd) 


Provide signature approval of the board report or prepare a 
minority report 


Review Presenters. The responsibilities of persons assigned to make 
presentations at each review are to: 

Assemble the data package for the review, including appropriate 
documents, and provide handouts of the oral presentation 
material and supplementary data as required 

Make the oral presentation and respond to questions 

Assist the cognizant manager in defining action itens 

Follow up and respond in writing on action itens assigned 
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CENTER NU. 1. 

DOCUMENT NO. 1-1 
REVIEW AND REVIEW ITEMS/DATA 

Project Requirements Review. As is the case with other level 2 
project reviews, the Requirements Review is formed and normally 
conducted by the standing Project Review Beard. 

Purpose. To assure that the project objectives (particularly the 
science objectives for a flight project) have been properly 
translated into definite and unambiguous statements of requirements; 
and to assure that the impact of these requirements on the design of 
the major project elements and systems is sufficiently well 
understood that trade-offs between requirements and constraints can 
be properly made. 


REYIfil-lIEMS- 

o Mission objectives 

o Science objectives 

o Mission design criteria 

o Project policy and requirements 

o Management structure 
o Constraints 
o Risk assessment 


Preliminary- Misaicn, ansL^ffitfias-ttsdat* 

Purpose. To assure that an adequate beginning has been made in 
designing the project systems to meet mission and science 
objectives. 


m i w .. 1T.EMS 

o Final draft of the Project Requirements document 
o Preliminary design documentation of the leading project system 
and conceptual designs of the other systems 
o Design approach 

o System interfaces and constraints 

o Developments required 

o Pro ject/sy steins block diagrams 

o Satisfaction of mission and science objectives 
o Problems and areas of concern 
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CENTER NO. 1 
DOCUMENT NO. 1-1 

REVIEW AND REVIEW ITEMS /DATA feont'd) 

Definition Plan Review 

Purpose. To ensure that the scope, resources, and schedule for the 
effort are compatible, that the technical direction and management 
structure are clearly understood and viable, and that any unusual 
circumstances or problems are identified. 


REVIEW ITEMS 

o Scope of effort 

o Schedule of definition phase activities with all critical 
milestones 

o Cost plan, by month 
o Workforce loading plan, by month 
o Procurements (studies, consultants) 
o Management structure 

o Identification of key personnel 

o Memoranda of Understanding with supporting Divisions 

o Proposed costing methods to be used in developing that portion 

of the Task Implementation Plan 
o Breadboard plans (if any) 

o Plans for tracking resource expenditure and technical 
activities during definition phase 
o Risk assessment 


Definition Results Review. 

purpose. To evaluate the adequacy of the draft plans, assess the 
state of readiness to proceed into the implementation phase, examine 
the validity of the costs presented, and recommend any changes to 
the plans prior to their release. 


REVIEW ITEMS 

o The results of studies carried out during the definition phase 
o The results of trade-offs carried out during the definition 
phase 

o The results of at least two different costing methods 
(detailed, analogy, model, or contractor quotations) 
o The. reconciliation of the differences between the results 
obtained fran the costing methods 

o Any changes to the proposal risk assessment memorandum (flight 
experiments) 

o Any changes to the proposal exceptions memorandum (flight 
experiments) 

The functional requirenents document draft 
The task implementation plan draft 
Risk assessment 


o 

o 

o 
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DOCuHtNT NO. 1-1 

REVIEW AND REVIEW ITEMS/DATA (cont'd) 

Implementation. Plan Review . 

Purpose . To assess all aspects of the Implementation Plan and to 
recommend remedial action, if necessary prior to its approval and 
implementation. 


REVIEW ITEMS 

o Pinal draft of the Implementation Plan 

o Any changes in cost plans imposed by the actual funding 
provided by the sponsor at the initiation of the implementation 
phase 

o Any changes in the scope imposed by the sponsor from the 
proposal or definition phase 
o Any changes in the schedule imposed by the sponsor 

o Any changes to the proposal risk assessment memorandum (flight 

experiments) 

o Listing of exceptions to requirements of JPL Document 750-150 
(flight experiments) 

o Implenentation organization and staffing 

o Schedule and resource plans including reserves and contingency 
plans 

o Test and training plans, criteria, deliverables , and methods 
o Compliance with or waiver from all standards 

o Risk assessment 

o Plans for reserve utilization 

If no Definition Results Review has been conducted, the following 
additional itens are presented: 

o The results of any studies carried out after the submission of 
the proposal or definition phase results 
o The results of any trade-offs carried out after the submission 
of the proposal or definition phase results 
o The results of at least two different costing methods 
(detailed, analogy, model, or contractor quotations) 
o The reconciliation of the differences between the results 
obtained from the costing methods 


User Requirements Review. This review is analogous with the project 
Requirements Review, and may also be conducted during the definition 
phase. 

Purpose. To demonstrate that the Task organization has attained an 
understanding of the requirements that the task is to fulfill. 
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DOCUMENT NO. 1-1 

REVIEW AND REVIEW ITEMS/DATA (cont’d) 

REVIEW ITEMS 

o Review of the user needs statement 

o The results of "get smart" studies 

o Verification of the desired capabilities through a solicitation 
of inputs from future users 

o The results of any trade-offs carried out as a result of new 
inputs 

o The reconciliation of differences between the Task plans and 
the user needs 

o Any changes in the technology mix or workforoe 

o Any changes to the proposal risk assessment, costs, or schedule 


Functional Design Review. 

Purpose. To ensure that the proposed technical direction of the 
task plan is conceptually viable and that any developments required 
for successful completion of the task are identified. 


REVIEW ITEMS 


o Configuration and definition of hardware and software elements 
o Design approach 

o Trade-offs for important design parameters 
o Satisfaction of subsysten requirements by the design 
o Areas of concern and problems 

o Interfaces with other subsystems (hardware and software) and 
cables 

o Man-machine interfaces and operations 
o Final draft of the Functional Requirements 
o Detailed end-to-end system block diagram 
o Rationale for systan technical approach 

o Subsystem budget apportionments (error, weight, power, 
reliability, etc.) 

o System interfaces and constraints 
o Areas of technical uncertainty 
o Developments required 

o Identification of any differences from proposal baseline 

o Design traceability 
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DOCUMENT NO. 1-1 


REVIEW AND REVIEW ITEMS/ DATA (cont'd) 


Preliminary Des ign Review . A large number of Preliminary Design 
Reviews are conducted during the life of a project or task at 
various levels of complexity and formality. The previously 
discussed Functional Design Review and Preliminary Mission and 
Systems Review are specific examples of top level preliminary 
reviews which may be required. 

Purpose. The Preliminary Design Review is a review at the system, 
subsystem, or assembly level and is intended to assure that the 
proposed implementation solutions satisfy the needs and requirements 
within existing technologies. This review may assess the timely 
availability of technically qualified personnel and proper 
facilities. 


REVIEW ITEMS 

o Final functional requirements 

o Canpliance with functional requirements/specifications 
o Suitability of inherited designs and hardware 
o Environmental design requirements 
o Interface design requirements 
o Design parameters, restraints, and constraints 

o Discussion of block diagrams 

o Discussion of single-point failures (flight hardware) 
o Preliminary parts, materials, and processes list 
o Hardware and software development plan and schedule (including 
verification tests on analyses to be performed) 
o Software standards to be applied 

o Experiment performance analysis, including an analysis of the 
instrument-system accuracy requirements (flight experiments) 
o Preliminary data management flow and reduction plans 
o Safety, reliability, and QA compliance plans 
o Plans for spares provisioning 
o Support equipment requirements and plans 
o Preliminary operations planning 

o Schedules and status of the task including cost as well as 
technical developments 
o Presentation of design optimization 
o Documentation plans and controls 

o Design traceability 
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REVIEW AND REVIEW ITEMS/ DATA (cont'd) 

Final Mission and Systems Design Review. This review is the top 
level Critical Design Review for a major project. See also Section 


Purpose. - To assure that -all of the final systems designs are 
compatible with the project objectives and mission and science 
requirements. It also reviews the overall macrostructure for 
Project systems integration and test. 


REVIEW- ITEMS 


o Inheritance assessment 

o Specifications (test and performance) 

o Prototype test results 

o Design mechanization and analysis 

o Design trade-offs and alternatives considered 

o Detailed interfaces and cable configurations 

o Detailed analysis of failures 

o Safety analyses and human factors 

o Maintainability, repairability, operability, and reliability 
o Schedule and resource plans 

o Man-machine interfaces 

o Training plans 

o Conformance of the design to the functional requirenents 
o Differences between the system and subsystem performances 
relative to the performances estimated at the preliminary 
design review 

o Status of preliminary design review action items 
o Detailed design parameters, restraints, and constraints 
o Interface details/agreements 

o Weight and power 

o Detail circuit drawings 

o Electronic parts classification 

o Screening specifictions 

o Nonelectronic parts, materials, and processing list 
o Purchased devices list 

o Materials and processing specifications 

o Identification and traceability 

o Electronic and mechanical parts stress analysis results 
o Worst-case aialysis 

o Conformance to environmental design requirements 
o Details of design, construction, and electronics packaging 
o QA plans and procedures 

o Manufacturing plans 

o Configuration control plans (hardware and software) 
o Documentation status 

o Flow plan and schedule status (hardware and software) 
o Problen/failure reporting system 

o Failure mode, effects, and analysis 

o Preliminary test plans 

o Developmental and detailed test results 

o Qualification and acceptance test plans 

o Calibration plan 

o Data management plan 

o Formal and informal report plans 

o Final design verifications plans 
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DOCUMENT NO. 1-1 

REVIEW AND REVIEW ITEMS/DATA (cont'd) 


REVIEW ITEMS 

o Initial customer handling or operating constraints and 
precautions 

o Spares provisioning 
o Support equipment requirements 

o Flight operations approach 

o Schedule and cost status 

o Risk assessnent 


Detailed (Critical) Design Review . A Detailed Design Review is 
conducted to match each Project or Task Preliminary Design Review. 
Due to high technology content or detailed technology, major 
projects often utilize pre-critical design reviews by individuals or 
teams. The results of these detailed studies are presented as a 
part of the Detailed (Critical) Design Review materials. 

Purpose. To assure that the detailed design meets the requirements, 
that an adequate qualification program exists, and that detailed 
interfaces are compatible. This review also determines to what 
degree the design and reliability meet the provisions of the system 
design characteristics, constraints, and other controlling 
requirements. 


REVIEW ITEMS 

o Inheritance assessment 

o Specifications (test and performance) 

o Prototype test results 

o Design mechanization and analysis 

o Design trade-offs and alternatives considered 
o Detailed interfaces and cable configurations 

o Detailed analysis of failures 

o Safety analyses and human factors 

o Maintainability, repairability, operability, and reliability 

o Schedule and resource plans 

o Man-machine interfaces 

o Training plans 

o Conformance of the design to the functional requirements 
o Differences between the system and subsystem performances 
relative to the performances estimated at the preliminary 
design review 

o Status of prel imin ary design review action items 
o Detailed design parameters, restraints, and constraints 
o Interface details/agreements 

o Weight and power 

o Detail circuit drawings 

o electronic parts classification 

o Screening specifictions 
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REVIEW AND REVIEW ITEMS/DATA (cont'd) 

REVIEW ITEMS 

o Nonelectronic parts, materials, and processing list 
o Purchased devices list 

o Materials and processing specifications 

o Identification and traceability 

o Electronic and mechanical parts stress analysis results 

o Worst-case analysis 

o Conformance to environmental design requirements 
o Details of design, construction, and electronics packaging 
o QA plans and procedures 

o Manufacturing plans 

o Configuration control plans (hardware and software} 
o Documentation status 

o Flow plan and schedule status (hard/are and software) 
o Problem/failure reporting system 
o Failure mode, effects, and analysis 

o Preliminary test plans 

o Developmental and detailed test results 
o Qualification and acceptance test plans 
o Calibration plan 

o Data management plan 

o Formed, and informal report plans 

o Final design verifications plans 

o Initial customer handling or operating constraints and 
precautions 

o Spares provisioning 

o Support equipment requirements 

o Flight operations approach 

o Schedule and cost status 

o Risk assessment 


Manufacturing Review. A detailed review of the procedures and 
facilities which will be used in the fabrication of flight hardware. 

Purpose. TO assure that the facilities and procedures used in the 
fabrication of flight hardware are adequate. Items of specific 
importance are the manufacturing orders and configuration control 
methods to be used. 


REVIEW ITEMS 

o Manufacturing orders 

o Configuration control plans 

o Manufacturing process 

o Personnel certification 

o Mandatory inspection points 

o Rework procedures 

o Review of manufacturing facilities 

o Hardware protection procedures 
o Production control plans 


D-17 



CENTER NO. 1 
DOCUMENT NO. 1-1 

REVIEW AND REVIEW ITEMS/ DATA (cont'd) 

80% Cofflplete/90-Dav Review. 

Purpose. To assure that the task commitments will be met and to 
present the plans for meeting those commitments within the time and 
resources remaining. 


REVIEW ITEMS 


o Status of effort remaining 

o Final report status and outline 

o Schedule of remaining activities 

o Workforce and resource plan for remaining effort 
o Reconciliation of remaining effort with remaining resources 
o Risk assessment 

o Plan for follow-on work 


Subsystem. Transfer Review . 


Purpose. To assure that the subsystem implementation has been 
completed, requirements have been met, required testing has been 
performed and passed, required documentation and spares are 
delivered, training support has been provided, and any discrepancies 
have been resolved. 


REVIEW ITEMS 


o Report on acceptance test performance 

o Spares status 

o Documentation status 

o Anomalies and liens 

o Training status 

o Software status 


Svstem Test Plan Review 

Purpose. To assess adequacy of planning for system level test and 
integration, and compatibility with project level design 
verification. 

REVIEW ITEMS 

o Verification of all functional requirements 
o Environmental requirements 
o Draft test plans 
o Test equipment design 

o Test canplex block diagrams 

o Test layout and circuitry 

o Test objectives 

o Open action items requiring test to close 
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REVIEW AND REVIEW ITEMS/DATA fcont'd) 


Svstem Te st Review 

Purpose. To review results from each system level test and to 
assure that any open action items relative to functional or 
environmental qualification are resolved. 


The review concentrates upon complete system test results 
records. It is also appropriate to review open action items and 
prior problem/failure history against test results. All functional 
requirements must be verified. 


Project Capabilitie s Review 

Purpose. To evaluate final prelaunch mission design and missi&ri 
capabilities against Project requirements; to compare "as-built" 
capabilities of all systems with established requirements; to 
prepare for intersystem testing; and to close out all open action 
items. 


REVIEW ITEMS 

o All system Test Review results 
o Mission design verification test plans 
o Intersystem testing plans 
o Problem/failure resort history 

o Status of all currently approved drawing and specifications, 
including specification changes and waivers 
o Hardware status and documentation, including configuration 
management records 


Preshirroent Review 

Purpose. To verify that testing is complete with no unresolved 
problems, and to evaluate readiness of the hardware for delivery. 
For a flight project, to verify the readiness of ground-based 
facilities at the launch site and of the integrated flight team 
training. 
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REVIEW AND REVIEW ITEMS/ DATA (cont'd) 


REVIEW ITEMS 

o Verification of availability of current approved drawings and 
specifications, and of adequate definition of the item, 
including approved specification changes and waivers 
o Review of hardware status and documentation, including all 
interface documentation, to assure that the latest changes have 
been incorporated in affected hardware, as reflected in a 
summary of the Engineering Change Requests 
o Review of test documentation status and determination (by 
review of the test results) of the adequate and proper testing 
of hardware 

o Status of prior formal review action item closures 

o Review of the Problem/Failure Report documentation to determine 

its status and evaluate its total impact, particularly with 
respect to risk for the experiment 

o Review of the quality history to ensure that necessary 
inspections have been performed and all discrepancies have been 
properly disposed of, including Material Review Board actions 
o Review of the results of each instrument calibration report to 
ensure compliance with the Functional Requirements and the 
calibration plan, including measured instrument weight and 
power 

o Shipping and handling requirements and plans 
o Safety compliance verification 

o Status and acceptability of the deliverable end item data 
package 

o Risk assessment 


Project Readiness Review. This review is required only for flight 
projects, and is normally chaired by the Project Manager. 

Purpose. To assess results of the ground data test program; verify 
that the flight crew is trained and that ground system deficiencies 
that might affect launch and encounter have been corrected; assure 
that operational procedures have been verified; and assess readiness 
of the flight sy stans after integration with the injection vehicle. 


This review examines any problems encountered after the 
flight systems have been delivered to the launch site, and assures 
that all integration and final testing have been satisfactorily 
completed. 
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REVIEW AND REVIEW ITEMS/DATA (cont'd) 


Launch Readiness Review . Like the Project Readiness Review, this 
review is unique to flight projects. It is normally chaired by the 
Assistant Laboratory Director for Flight Projects. 

Purpose. To verify that no unresolved problems exist with the 
flight systems, launch vehicle, launch complex, and operational 
configuration for launch; and to commit to launch. 


REVIEW ITEMS 

Same as for the Project Readiness Review, including the 
findings of that review. 


Postlaunch Software Requirements Review 

Pur pose. To review adequacy of software requirements and planning 
for encounter operations. 


REVIEW ITEMS 


o All software documentation 
o Current ope rati oral experiences 

o Planned encounter scenarios 


Orbit Operations Readiness Review. This review is required only for 
a flight project which involves operations in orbit about a body in 
space. A similar review would be required for a flight project 
which involved landed operations. 

Purpose . To verify operational readiness for encounter. 


REVIEW ITEMS 


Includes as a minimum all plans for orbit insertion and 
science sequences in the vicinity of the objective. 
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REVIEW AND REVIEW ITEMS/ DATA (cont'd) 


Encounter Capabilit ies Review. This review is required only for a 
flight project when final refinement of operational planning 
requires information only available after the spaoe systems are in 
orbit, e.g., precise determination of trajectories for planetary 
satellite encounters or orbits. 

Purpose. To review any changes in mission operations planning to 
assure that the mission as implemented will meet project 
requirements; to assess the status of ary capabilities being added. 


REVIEW ITEMS 


All operational planning and science sequence changes. 


Final Report Review. This discussion addresses primarily the review 
of a task rather than a project. 

Purpose . To review the results of the project or task, in 
particular the final reports, to ascertain that the report is 
credible and of a sufficient quality to satisfactorily represent 
JPI/Caltech. 


REVIEW ITEMS 


o Comments on the final report draft 
o Customer viewpoint on task 
o Potential for follow-on work 
o Schedules and resources for retaining effort 
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Formal Reviews . During the review, requests for action are 
submitted by review board members and attendees if invited to 
participate on forms that are provided by the secretary. Following 
the review these are screened by the review board to consolidate 
them and to ensure that the chairman and cognizant manager 
understand the intent of the requests. The requests for action 
support the findings of the board and are normally included as an 
attachment to the board report. 


Informal Reyiew Minimum Level. An attendee may be designated to 
compile the requests, which may be provided either verbally or in 
writing. 
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Formal Reviews. The responsibility for accepting, rejecting, or 
acting in part on the board's recommendations rests with the 
cognizant manager. A written response to the board provides 
documentation of the disposition of the recommendations and provides 
assurance that the board's recommendations are properly interpreted 
and considered. The cognizant manager reviews proposed action itsns 
with those to whom the action is assigned. They should establish an 
action plan (i.e., how the action item will be resolved, by whom, 
when). The cognizant manager should report on the status of the 
action items to the convening authority, with copies to the board 
secretary and the Office of Engineering and Review. 

Formal Reviews. Action items are closed by memorandum, and copies 
are sent to the chairman for approval and distribution. The 
memoranda are to include the supporting information for the 
recommended closure action. The secretary of the board maintains a 
file of all board actions, including closure memoranda, as a central 
point for all board members. 

Formal Reviews. Following appropriate board deliberation and 
consensus where possible, the findings of the board and any 
recommendations are prepared by the secretary, approved by the 
board, and sent to the convening authority and the cognizant manager 
for their study and disposition. Dissenting opinions may also be 
attached. The cognizant manager determines disposition of the 
findings and recommendations; eg., identifies action items, 
formulates an ad hoc study team, makes policy decisions, or takes 
whatever action he believes is appropriate for each specific 
finding, concern, or recommendation. 


Informal Review Minimum Level. The cognizant manager advises 
the convening authority by issuing a list of action items, action 
item assignees, and due dates. 

Informal Review Minimum Level. At preestablished intervals the 
cognizant manager should update the listing of action item closures 
and obtain new closure dates on past-due items. This information is 
distributed periodically until all action items are closed. 


Informal Review Min imum Level. A brief memo documenting findings is 
desirable. 
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LAUNCH VEHICLE COMPONENT AND SYSTEM 
DESIGN REVIEW 


Purpose 

This section establishes general guidelines and requirements for con- 
ducting launch vehicle component and system des-ign reviews which are un- 
der the purview of NASAcenteras applicable to the Atlas/Centaur launch 
vehicle and its related ground support equipment. It is not intended 
that these reviews be held for every vehicle, but rather for new systems 
designs, first of a kind components and/or changes in design philosophy. 

Scope . 

The scope of the design reviews may range from piece parts, subassem- 
blies, unit level, systems or combined systems, on vehicle airborne or 
ground support hardware, and be mechanical or electrical in nature or com- 
binations of both. Since the scope is wide, a list of common guidelines 
is indicated for design- reviews in general. A second list indicates those 
criteria which may apply to a particular hardware design and should be re- 
ferred to as a checklist for possible areas to be covered. 


PROCEDURES 


Procedures are not described in this document. 
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' NASA/ center Chairman 

LV Division Chief - Optional 

LVD 8ranch Chief/Section Head as appropriate - Optional 
Responsible System Engineer 
System Engineering Staff Engineer 
Mission Project Engineer - Optional 

Reliability and Quality Assurance Engineers - Optional 
Con t rac to r s /Ven do rs 
KSC/EV - Optional 
Spacecraft Agency - Optional 


RESPONSIBILITIES 


The responsibilities of review board personnel are not 
described in this document. 


\ 


D-27 



CENTER NO. 2 
DOCUMENT NO. 2-1 

REVIEW ITEMS/ DATA 


1. Special environmental criteria; e.g., temperature, thermal 
cycling, humidity, radiation, pressure, acoustic, shock or steady state 
loads, EMI, acceleration. 

2. Size/weight considerations. 

3* Mounting requirements. 

A. Power, grounding or shielding requirements, connector locations, 
PIN assignment, etc. 


5. I terns for special consideration or urogram control; e.g., con- 
tamination, age/cycle/use limits, warning/caution marking, radiation, 
vibration limits, vibration effects and electrical and mechanical isolation. 


6. Packaging requirements. 

7. Hardware maintainability; ease of cleaning, adjusting and replace- 
ment . 


tion. 


8. Human engineering aspects. 

9. Special tools, test equipment and facilities required for fabrica- 
test or operations. 

10. New processes, materials or techniques required. 


11. Part and material selection of vendor items to optimize reliability, 
standardization, availability, inspection, test, handling, preservation and 
application. 


12. Testing at piece part, subassembly, unit and system levels. 

13- Special analysis required; e.g., circuits, thermal, stress, elec- 
trical stress/derating, clearance, worst case, tolerance, test logic, error, 
single point failure, hazard, etc. 

|1». Effect on other systems or interfaces. 

15. Effect on software. 

16. Identification methods. ' 
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1. Review and approve the design requi rements and criteria speci- 
fied for the hardware under review. 

2. Review and approve primary conceptual design and alternate 
designs considered. 

3- Review and approve general layout or block diagram level 
description of primary design. 

k. Review detail drawings/schemat ics where appropriate to explain 
design features. 

5. Review expected performance parameters, if appropriate. 

6. Review the design for failure modes and their effects on reli- 
ability, inspection, test and operation. The techniques of Failure Modes 
and Effects Analysis (FMEA) are used iteratively during design and develop- 
ment . 


7. Review design checklist to assure that all aspects of design 
have been covered as applicable. 

8. Review the design for its effects on personnel and launch vehicle 

safety. 

9. Review and approve the test program outlined for carrying the 
design through development, qualification and finally hardware acceptance 
testing. 


10. Review the design to see that it permits and facilitates pro- 
ducibility, repeatability and inspectab i l i ty and that related quality con- 
siderations are obtained. 

11. Review the schedule for design release through hardware avail- 
ability. 

12. Review cost, contractual status and/or vendor selection. 
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No feedback process is described in this document. 


CLOSEOUT 


The contractor provides the necessary materials to present the design in- 
cluding drawings, schematics, photographs, Vu-graphs/s 1 ides and data. The 
contractor will also prepare the minutes and publish any action items 
resulting from the review with signature concurrence on the action items. 

Due dates for satisfying the action items shali be estab 1 i shed and 
closed as soon as possible. The status and completion of action items will 
be monitored by the responsible system engineer. 
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CONCEPTUAL DESIGN REVIEW (CODR) 
PRELIMINARY DESIGN REVIEW (PDR) 
CRITICAL DESIGN REVIEW (CDR) 

PREFLIGHT REVIEW PRESHIPMENT READINESS REVIEW 

POST FLIGHT REVIEW 
PROCEDURES 

This instruction sets forth the criteria, policy, 
and procedures for review of aeronautic and space 
flight projects and experiments under the management 
of this Center and flight experiments included as a 
part of a mission managed external to this Center, 

The criteria stated herein are minimum requirements 
that will permit total visibility by a third-party 
review committee of the technical and management 
aspects of • - : flight 

projects and experiments. The requirements of this 
instruction do not supersede other reviews impose^ 
by NASA Headquarters, other Centers, or other 
reviews. 

This Center will conduct the following sequential 
set of reviews for flight projects and experiments: 

- Design Reviews 

. Conceptual Design Review (CoDR) 

. Preliminary Design Review (PDR) 

. Critical Design Review (CDR) 

- Preshipment Readiness Review 
Preflight Review 

- Postflight Review 

This policy may be altered by the cognizant Program 
Director to meet unique requirements. 

The above reviews are advisory in nature and do not 
relieve personnel of the responsibility for the 

success of the project or mission. 

The Chairperson of each review, or the cognizant 
Program Director, may also establish other special 
reviews to supplement the above reviews. 


All Program Directors are to support the above 

reviews. They are to furnish senior personnel, 
experienced in the required technical disciplines, 
as requested. 

The Chairperson of each review shall organize the 
Committee and draw support from other Centers or 
Federal Agencies as required. 
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Deviations from the review policies established in 
this instruction may be made only with the approval 
of the cognizant Program Director. 

The primary objective of the above reviews is to 
enhance the probability of success of flight 

missions. This will be achieved by utilizing the 
cumulative knowledge and skills of the team of engi- 
neers and scientists who have been selected for 
their experience with particular systems and 
functions. 

The reviews will be technically oriented, however, 
proper consideration is to be given to operating 
constraints — particularly those involving primary 
mission objectives and program costs and schedules. 

The Project Manager, or equivalent, is to insure 
that personnel from the appropriate NASA 
Headquarters Program Office are Informed of review 
date and, to the extent practicable, arrange for 
reviews to be scheduled to allow NASA Headquarters' 
participation. 


There are three levels of design reviews which are 
to be phased into the project/mission schedule at 
appropriate times. New designs are to Include all 
three reviews; namely: 

. Conceptual Design Review (CoDR) — In some 
cases this review may be eliminated if the 
concept has been demonstrated and only minor 
changes are necessary. 

. Preliminary Design Review (PDR) 

. Critical Design Review (CDR) 

The CoDR will normally be scheduled at a time which 
will permit assessment as noted above and prior to 
the start of development testing. 
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CDR, PDR, CODR, POST FLIGHT REVIEW 
PERSONNEL AND RESPONSIBILITIES 


Assistant Director for Systems 
Engineering and Operations or 
designated representative 
Appointed by the Chairperson 
Representative, Systems Safety, 
Quality and Reliability Office 
At least five representatives from 
supporting line organizations as 
deemed appropriate by the 
Chairperson. 


The Project Manager, or equivalent, is respon- 
sible for notifying the Chairperson of a review 
within sufficient time so as to permit an orderly 
preparation of the review. 


Chairperson: 

Secretary: 
Members : 


PRESHIPMENT READINESS REVIEW 


Chairperson: Chairperson, CDR Committee 

Secretary: Appointed by the Chairperson 

Members: Head, Systems Safety, Quality and 

Reliability Office 
At least five representatives from 
supporting line organizations as 
deemed appropriate by the 
Chairperson 


PREFLIGHT REVIEW 

Chairperson: Cognizant Program Director 

Secretary: Appointed by the Chairperson 

Members: Chairperson, Preshipment Readiness 

Review 

Chief, Flight Electronics Division 
Chief, Systems Engineering Division 
Public Affairs Officer 
Head, Systems Safety, Quality and 
Reliability Office 
Representatives from other organi- 
zations as deemed appropriate by 
the Chairperson. 
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PERSONNEL AND RESPONSIBILITIES (Contd.) 
PREFLIGHT REVIEW (Contd. ) 


The Chairperson is responsible for documenting the 
major findings of the Preflight Review resolution of 
all open Items, and reporting flight readiness of 
the project/mission to the Director center. 


POST FLIGHT REVIEW 

The Project Manager, or equivalent. Is to arrange 
for such a review through the cognizant Program 
Director* 
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CONCEPTUAL DESIGN REVIEW 

The objective of the CoDR is to present the scien- 
tific requirements and examine the proposed design 
approach to accomplish these requirements by 
examining, completing, and presenting the: 

- initial design 
trade-off studies 

- alternate configurations 

- selection of critical parts 

- preliminary analyses 

- definition of environments 
interface requirements 

- Government Supplied Equipment (GSE) 

- project planning 

- software approach, etc. 

so that the design can be examined at the conceptual 
level with confidence. 

, PRELIMINARY DESIGN REVIEW 

The objective of the PDR is to examine in detail the 
baseline project/mission design being planned for 
manufacture and qualifications, validate the design 
approaches as related to project/mission require- 
ments, and cover such items as: 

- mechanical design layouts 

- circuit designs 

- design analyses 

- performance analyses 

- results of development testing 

- GSE requirements 

- manufacturing and qualification test planning 

- status of reliability, quality assurance, and 
systems safety programs 

- status of planning for data retrieval, analysis, 
and publication 

The PDR will normally be scheduled at the completion 
of the preliminary design and prior to the fabrica- 
tion of qualification hardware. 
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CRITICAL DESIGN REVIEW 

The objective of the CDR is to examine details of 
the final design and mission, verify the final 
plans, design, fabrication plans, and flight accepc- 
tance test planning as related to project/mission 
requirements, and cover such Items as: 


- updated final design and analyses 

- qualification test and calibration test results 

- functional and performance test results 

- reliability, quality assurance, and safety 
programs 

- a review of the plana for data retrieval, 
analysis, and publication 

The CDR will normally be scheduled at the completion 
of the qualification test program. 


PRESHIPMENT READINESS REVIEW 

The objective of the Preshipment Readiness Review is 
to inspect flight hardware, make final review of 
project/mission plans, concentrate on results of 
acceptance testing, and cover such items as: 

- validating the quality of the hardware 

- confirming that the hardware is flightworthy and 
will perform properly under the simulated flight 
environment 

- assessing that the mission objectives will be met 

- compliance with mission requirements and 
specifications 

- refurbishment and recalibration plans (when 
required) 

- shipping and storage plans 

- confirmation of compatibility with all interface, 
weather protection, and contamination control 
plans 

all failures or anomalies during test 
“ status of safety and reliability analyses and 
verification of compliance with documentation 
requirements 

The Preshipment Readiness Review will normally be 
held after the completion of major acceptance 
testing and prior to shipment of flight hardware. 
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PREFLIGHT REVIEW 


The objective of the Preflight Review is to assess 
the overall readiness of the project to perform its 
project/mission and is to cover such items as: 

- flight readiness of all hardware, software, and 
operational elements 

- completed ground support operations 

- interfaces with other flight equipment 

- ground based mission support requirements 

- flight operations plans 

- data retrieval and processing. Including ground 
network compatibility tests 

- public information plans 

The Preflight Review will normally be held as near 
as practicable to the flight date. 


POST FLIGHT REVIEW 

A Postflight Review for Center Management is to be 
held as soon as a general assessment of the 
project/mission can be made. 
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Minutes 

Minutes of each review are to be distributed to 
committee members and appropriate Division Chiefs 
and Program Directors. 

Minutes are to be made a part of project/mission 
files. 
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CLOSEOUT 


The Project Manager, or equivalent, is responsi- 
ble for Insuring that appropriate action Items, 
comments, and recommendations resulting from a 
review, also any open action items, are 
appropriately discussed at the next review. 


At the conclusion of the CDR, the review committee 
should have confidence that they have reviewed the 
final design and planning, that sufficient data has 
been presented to demonstrate the design is com- 
patible with the total flight system, and the 
project/mission will be accomplished within cost and 
schedule constraints. 
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BASELINE DESIGN REVIEW 


PURPOSE 

To provide policy, guidelines, and responsibilities for a consistent approach 
to the conduct of baseline design reviews research and development 
hardware and software. 

SCOPE 

This Instruction is applicable to all involved in the plan- 
ning, research, design, development, testing, and management of space hard- 
ware and software, subject to the limitations as noted in the Instruction. 


PROCEDURES 


It is the policy that a consistent approach be used in the preparation 
and conduct of baseline design reviews for all programs, projects, payloads, 
and experiments managed by the Center. In consonance with this policy, it is 
recognized that the diversified programs and projects assigned re- 
quire considerable variation in the management and operational modes used by 
the program/project managers. Therefore, although a consistent approach to 
baseline design review is essential, the scope of the individual reviews may 
vary. For example, in the development of experiments, a limited scope of 
review may be proposed and conducted, subject to agreement and concurrence of 
the program/project manager and the Director, Science and Engineering. For 
full scale reviews the complete Board Structure and documentation as speci- 
fied by this Instruction will be required, consistent with the scope of the 
review, and subject to the agreement and concurrence of the program/project 
manager and the Director, Science and Engineering. 
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BASELINE DESIGN REVIEW 
PERSONNEL 


BOARD 


. PROGRAM/PROJECT MANAGER 
. PROGRAM/ PROJECT OFFICES LEVEL 
. OFFICE OF DIRECTOR LEVEL 
. CHIEF ENGINEER 
. LAB DIRECTORS LEVEL 

. SAFETY OFFICE AND OTHERS AS APPLICABLE 
. OTHER NASA 
. OTHER GOVERNMENT 
. CONTRACTOR (NON-VOTING) 


PREBOARD 


. PROGRAM/PROJECT OFFICES LEVEL 
. CHIEF ENGINEER 
. LAB DIRECTORS LEVEL 
. OTHER LAB PERSONNEL 

. SAFETY OFFICE AND OTHERS AS APPLICABLE 
. OTHER NASA 
. OTHER GOVERNMENT 
. CONTRACTOR (NON-VOTING) 


TEAM 


. PROGRAM/* 5 ROJECT OFFICES LEVEL 
. CHIEF ENGINEER 
. OTHER LAB PERSONNEL 

. SAFETY OFFICE AND OTHERS . AS APPLICABLE 
. OTHER NASA 
. OTHER GOVERNMENT 
. CONTRACTOR (NON-VOTING) 
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BASELINE DESIGN REVIEW 
PERSONNEL (cont‘d) 


TEAM 


PROGRAM/PROJECT OFFICES LEVEL 
OTHER LAB PERSONNEL 

SAFETY OFFICE AND OTHERS AS APPLICABLE 
OTHER NASA 
OTHER GOVERNMENT 
CONTRACTOR (NON- VOTING) 
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Program/Project Offices are responsible for the initiation and overall 
conduct of design reviews and for providing a summary of each review and 

the results to management. This responsibil ity includes 
preparation of a Configuration Management Plan for 

each discrete program/project. The Configuration Management Plan will 
establish minimum requirements for each review.- the required data and 
the completion percentages of the data will be included 
in the Configuration Management Plan following approval. Pro- 
gram/project managers will obtain the required concurrence from the As- 
sociate Director for Engineering, S&E. Resolution of any nonconcurrence 
will involve joint review and agreement by senior S&E and Program Office 
management. 

• Science and Engineering is responsible for the technical aspects of 
design reviews, for providing technical support and expertise at all 
levels of the review, and for presenting assessments to the Program/ 
Project Offices. 

• Review Boards will review previous actions by teams and Preboards and 
rule on (approve/disapprove) RID action recommendations submitted by the 
Preboard, assuring that the design requirements levied on the program/ 
project are met and verified, and recommend approval of the baseline 
documents. 

• Preboards will review previous actions by teams; evaluate RID's from 
various technical teams; integrate or consolidate similar RID's; recom- 
mend acceptance of RID's as written or modified; recommend RID's for 
study; recommend RID's to the Board for resolution; and provide a summary 
of actions to the Board. 

• Teams will evaluate RID's; integrate similar RID's from subteams; gen- 
erate RID's; recommend disposition of RID's to Preboard; and provide a 
summary of actions to the Preboard. 

® Subteams will review and evaluate data made available for review; and 
generate RID's. In cases where subteams are not required, these respon- 
sibilities will be carried out at the team level. 
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REVIEW ITEMS/DATA 


. SCIENCE REQUIREMENTS LISTING 
. DRAWINGS, SKETCHES AND SCHEMATICS 
. ANALYSES 
. STUDY REPORTS 
. DESIGN CONCEPT DOCUMENTS 
. PRELIMINARY INTERFACE REQUIREMENTS 
. REQUIREMENTS SPECIFICATIONS 
. PRELIMINARY WBS 
. GENERAL TEST PLANS 
. PRELIMINARY R&QA PLANS 
. PHASE B STUDY RESULTS 
. DEVELOPMENT PLANS 
. FLOW DIAGRAMS 
. SAFETY ANALYSIS 
. SAFETY ANALYSIS 
. FMEA AND CRITICAL ITEM LIST 
. TEST VERI FI CAT I ON/ VALIDATION PLANS 
. INTERFACE REQUIREMENTS 
. PART 1 CEI SPECIFICATION (DRAFT) 

. SYSTEM DESCRIPTION DOCUMENT 
. WBS 

. SOFTWARE PROGRAM DESIGN DOCUMENT 
. SPARES PHILOSOPHY 
. ICOs 

. TEST RESULTS (PRELIMINARY) 

. CODE TO DESIGN SPECIFICATIONS 
. USERS MANUALS 
. ACCEPTANCE TEST 
. SPARES LIST 
. INTEGRATION PLANS 
. SUBSYSTEMS DESCRIPTION DOCUMENT 
. ALL TECHNICAL DATA, DRAWINGS, AND SPECIFICATIONS 
. DELIVERABLE DATA PER CONTRACT 

. DATA AS DEFINED IN PACKAGE OF DES REVIEW ANNOUNCEMENT 
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REVIEW ITEMS/DATA (cont'd) 


VERIFY CONFIGURATION CONCEPTS AND REQUIREMENTS 
VERIFY MISSION OBJECTIVES 
QUALIFY APPROACH 

ESTABLISH AND APPROVE PROGRAM/PROJECT REQUIREMENTS BASELINE 

BASIC DESIGN APPROACH 

COMPATIBLE DESIGN/REQUIREMENTS 

TEST PLANNING 

SAFETY ASSESSMENT 

ESTABLISH AND APPROVE DESIGN REQUIREMENTS BASELINE 

CONFIGURATION DESIGN 

SYSTEMS COMPATIBILITY 

DESIGN INTEGRITY 

RELIABILITY ASSESSMENT 

ESTABLISH SCHEDULE 

ESTABLISH AND APPROVE "DRAWING BASELINE" 

CONFIGURATION WAIVERS AND DEVIATIONS 
VALIDATE QUALIFICATION TESTS 
ASSESS SYSTEM TESTS 
ACCEPT GSE CONFIGURATION 

ESTABLISH AND APPROVE PRODUCT CONFIGURATION BASELINE 

REVIEW CRITICAL HARDWARE 

REVIEW PARTI CAL 

ASSESS DEFICIENCIES 

CERTIFY DESIGN 

CERTIFY READINESS 
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FEEDBACK AND CLOSEOUT 


Review Boards will review previous actions by teams and Preboards and 
rule on (approve/disapprove) RID action recommendations submitted by the 
Preboard, assuring that the design requirements levied on the program/ 
project are met and verified, and recommend approval of the baseline 
documents. 

Preboards will review previous actions by teams; evaluate RID's from 
various technical teams; integrate or consolidate similar RID's; recom- 
mend acceptance of RID's as written or modified; recommend RID's for 
study; recomnend RID's to the Board for resolution; and provide a summary 
of actions to the Board. 

Teams will evaluate RID's; integrate similar RID's from subteams; gen- 
erate RID's; recommend disposition of RID's to Preboard; and provide a 
summary of actions to the Preboard. 

Subteams will review and evaluate data made available for review; and 
generate RID's. In cases where subteams are not required, these respon- 
sibilities will be carried out at the team level. 
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DOCUMENT NO. 5-1 

DESIGN^ ENGINEERING SHUTTLE TRANSPORTATION 
SYSTEM READINESS REVIEW CYCLE 


PURPOSE 

This directive establishes requirements, organizational responsibilities 
and procedures to be used within DE for the preparation of the Launch 
Readiness Review cycle presentations. 
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PROCEDURES 


Readiness Reviews (LRR) 

The following reviews are required: 

(a) ET/SRB Mate Review 

(b) Pre-Stack Review 

(c) VAB Rollout Review 

(d) Launch Readiness Review 


D-51 


CENTER NO. 5 


DOCUMENT NO. 5-1 
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. Project Management Directorate, DF 

(1) Prepares LRR/Pre-LRR presentations. 

(2) Disseminates schedules and program requirements within OE 
for Readiness Reviews. 

(3) Provides representations at all Readiness Reviews. 

(4) Establishment of badglng requirements and distribution. 

. Mechanical and Facilities Engineering Directorate (DO) and the 
Electronic Directorate (DL ). Provides design support to DF as 
required In the development of Readiness Reviews. 

. Management Operations Office, DE-MOO 

(1) Develop policy for Readiness Reviews. 

(2) Coordinate with other elements to ensure that DE 

policies and procedures are compatible wltn . policy. 
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The Readiness Review Cycle typical coverage will Include: 

• major configuration changes subseauent to 
previous launch. 

• - previous anomalies, damage or problems as 
related to the activity under review. 

Present the status of priority spare parts. 

• the Systems Assurance Analyses (SAA's). List 
the numbers of category 1, IS and 2 single failure 
points (SFP's) and residual hazards and discuss those 
for which the risks have not been accepted. 

• a status of the Operation and maintenance 
Requirements and Specifications Documentation (OMRSD) 
and acceptance of waivers received from 0AM. 

• a summary assessment of the readiness of the 
design of the hardware and software to support 
achievement of the objectives of the milestone event. 

• ' a listing of all approved changes not yet 
installed, that are required to support the milestone 
event. 
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• Review major configuration changes subsequent to 
previous launch. 

•Review previous anomalies, damage or problems as 
related to the activity under review. 

• Present the status of priority spare parts. 

• Review the Systems Assurance Analyses (SAA's). List 
the numbers of category 1, IS and 2 single failure 
points (SFP's) and residual hazards and discuss those 
for which the risks have not been accepted. 

• Present a status of the Operation and maintenance 
Requirements and Specifications Documentation (OMR SD) 
and acceptance of waivers received from 0AM. 

• Present a summary assessment of the readiness of the 
design of the hardware and software to support 
achievement of the objectives of the mflestone event. 

• Provide a listing of all approved changes not yet 
Installed, that are required to support the milestone 
event. 
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SPACE TRANSPORTATION SYSTEM 
FLIGHT READINESS REVIEW.' 


PURPOSE 

This Directive establishes the role of the Center in support of the 
Orbital Flight Test (OFT) Program flight readiness review (FRR) cycle. 


APPLICABILITY AND SCOPE 

This Directive applies to all organizational elements and serves as the 
authority document for preparation and conduct of the Flight Readiness 
Review (FRR). It also defines the Center's responsibilities in support 
of the National Aeronautics and Space Administration (NASA) Office of 
Space Transportation Systems (OSTS ) OFT FRR. However, this Directive 
recognizes that the Payload Mission Managers are responsible for the 
individual mission payloads and they will execute "Certification of 
Flight Readiness" documents and FRR functions which are outside the 
scope of this Directive. 
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PROCEDURES 


The. FRR. cycle provides a means of assessing the readiness of flight 
harjware arid supporting (hardware/sol tware) systems to successful iy 
accomplish assigned functions during launch-, flight, and 'landing 
operations of the space vehicle. Cargo, and flight crews for each 
orbital Flight Test (OFT ) flight. 


An FRR will be scheduled approximately two weeks prior to each OFT 
flight. All activities/elements necessary tor safe and successful 
conduct of the forthcoming flight will be reviewed at each FRR as 
fol lows : 

. Pre-FRR Level II Reviews. Prior to each FRR, appropriate Level II 
reviews covering all elements associated with the upcoming flight 
will be scheduled and conducted at locations designated by Level 
II. Results of the Level II reviews will be documented and pro- 
vided to the FRR board members no later than one dav prior to the 
FRR. This documentation should include minutes of the proceed- 
ings, review item dispositions (RTDs), and signed statements of 
flight readiness for applicable categories. 

. FRR Conduct. FRRs will be chaired by the Assoc’ate Adininistra- 
tor/Space Transportation System (AA/STS), and board members will- 
be designated by him. Routine status, background and/or subsys- 
tems descriptions will be limited to those necessary for adequate 
coverage of the issues on the agenda. The agenda will be develop- 
ed by Level II, in coordination with the Program Director, to 
emphasize significant issues and open items identified at or sub- 
sequent to Level II reviews. 

. RuSt-FRR. All action items ana open work identified a: tne F Rkx 
must be formally closed prior to flight with the signature of the 
Program Manager and Program Director or their designated represen- 
tatives. The closeout documentation is to include a brief summary 
narrative description of the actions ta^en on each FRR action item 
and the documentation neertssary to support the summary narrative 
ie.g., vugraph material). lr. addition, open items that arise sub- 
sequent to the FRR that could delay the flight, change test vehi- 
cle conf iguration, or cause basic changes to the operations plan- 
ning are to be reported to the Program Director. 


Prior tu each launch, the level II Program Manager will review at', i 
, resent closeout rationale fur all open it ms at an ;.-2 day meet- 
ing with the Program Director and the A A/bTb. 
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The normal preparation _ in support of a FRR shall bey in by a 

flight readiness reviev/ ( FRR) which will be scheduled ‘j days prior 
to, and in preparation for, the Pre-FRK. The FRR shall be chaired 
oy tne Manager, Shuttle Projects Office and presented to tiie Center 
director. The review will describe and document the readiness state 
of the vehicle, USE, vehicle cargo, supporting systems and tacili 
ties, and launch, landing, and recovery teams to successfully accom- 
plish launch of the STS vehicle in support of oFTs. 

A Special FRR for STS-0 will be required due to the uniqueness of 
tne FRF as part of the CDOT for i'KWr and snail r>e conducted by the 
Manager, Shuttle Projects Office. This review -will describe a. id deco 
:.ent the state of STS-U readiness for successfully accomplishing the 
CDDT/FRF. A subsequent Delta Fi<R will be accui ip I i shed for STS-i. and 
it will be controlled within the management structure 
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PERSONNEL AND RESPONSIBILITIES 


The Manager, Shuttle Projects Office is responsible forr^-^-^^" - 

u_ Serving as the intercenter interface with NASA Headquarters 
and other centers for support of the FRR. > . 


. Receiving, interpreting, and disseminating FRk requirements, 
guidelines, agenda, and schedule. 

.• Providing a representative to serve as a member of the FRR Secre- 
tariat to arrange for: 

- Meeting facilities. - 

- Access controls. - : - 

FRR Secretariat support, Ho include office space, furniture,” 
supplies, clerical assistance, reproduction facilities, and 
telephones.. ' 


Reporting to Level I i 11 With the assistance of the Di recto rs~ of ~ 
Shuttle Operations, Design Engineering, Technical Support, Cargo 
Operations, Administration and Management Operations, Safety, R4QA 
and Protective Services, the Manager,; Cargo Projects Office, and 
the Biomedical Office on the readiness of the space vehicle, GSE, 
Shuttle to Cargo Integration,, launch and landing facilities, 
support, services, and .ETR. support; services as follows: 


- - Presenting data on:;.: / 



•; Status of vehicle and launch support facilities, sys- 
tems, equipment, ramie support, launch commit criteria, 
and other routine operational support (e.g., personnel 
training, documental lun, etg.). , 


t Category 1, IS, and 2 SFPs residual Hazards end problem 
reports. 


\ Deviation/Waivers. 

I Operation and Maintenance Instructions (OMls) status. 


; Other data deemed ;;ertinent to, or requested by, OSTS to 
be presented to Levels I and II. 
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^Certifying the readiness of the space vehicle, faci- 

lities, systems, equipment, support, launch and landing sup- 
port, and team services to support launch and landing for 
each fl ight 

Chairing the . FRR. 

* Developing guidelines and coordinating requirements with the 
Directors of participating primary organizations. 

* Identifying items which require resolution prior to launch, 
and assigning these items to the coynizant Director. 

* Providing the following facilities, arrangements, or 
services: 

- Date of the FRR and agenda. 

- Meeting facilities, including audio-visual equipment. 

- Access control and seat allocations. 

— Secretariat to riecord FRR action items. 

I . • 


The Director of Design Engineering is responsible for: 

. Presenting an assessment of 1, 1R, IS & 2 criticality categories 
as defined by the Facilities, Systems and Equipment Organizational 
Level OMD Baseline and essential to the support of launch and 
landing operations. 

• Presenting, in summary format, a listing of all Cateyory 1, IK , 

IS, and 2 single failure points (SFPs) and residual design iiazards 
(KDHS) that have been identified. The list should indicate those 
items for which the Center Director has accepted risks and nates 
of acceptances. (This includes system, line replaceable units 
(LRUs), and components.) 

. Discussing the status of software under his cognizance and associ- 
ated residual hazards. 

. Summarizing significant changes in conf: (jurat ; on of launch support 
facilities and systems baseline. 

. Discussing open mandatory nod i f ications , identifying those fur 
which modification packages are to be released, and providing im- 
pact of modifications upon the launch operations vehicle and sys- 
t ems . 

. Identifying and discussing any significant problems which could 
impact the launch schedule. 
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The Directors of Shuttle Operations and Technical Support are responsible, 
for presenting their respective assessments of facilities, systems, and 
equipment including associated hardware (within their OoM responsibility) 
as defined by the Facilities, Systems and Equipment Organizational Level 
0M0 Baseline and essential to launch and landing operations. The Director 
of Shuttle Operations is also responsible for presenting an assessment of 
_the flight hardware/ software relative to the launch and landing operations 
function. The presentations shall include: 

. Open work and' special tests to be completed (e.g^.OMIs, Cateyory 
G paper. Operations <£ Maintenance Requi remenis/Speci fi cat ions 
(OMRS) not met. Engineering Support Requests (ESRs), and concerns 
related to their successful completion). 

. : » w 

. Open or potential problems documented by problem report(s), devia- 
tion/waiver(s) or equivalent report(s). The discussion should 
identify operational concern, corrective action in work cr 
planned, and impact on the launch schedule. 

. Status of the launch commit criteria and a discussion of any 
changes (deltas) to the previous mission baseline. 

. Identification and status of any essential facility, system, or 
equipment not operational due to shortage of spare parts, includ- 
ing discussion of any work-arounds. 

A discussion of significant changes in the COOT and launch count- 
down UMIs. 

. Status of other operations and support functions, to include, but 
not be limited to, the following: 

- Range support. 

- Personnel training. 

" Replacement of limited life components. 

- Mater i a L shortages. . 

- Documentation. 

. Status of operational software. 

A summary of all residual operational hazards including those 
associated with the operational software. 

The Director of Shuttle Operations will sign the Certir i cat ion cf 
plight Readiness, Endorsement 02 . '■ - 

Tne Director of Administration and Management Operations is responsi- 
ple for presenting an assessment of the availability of spare parts 
for support of essential facilities, systems, and equipment reported 
nonoperational . 
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I ne Director, Safety, R&QA, and i’ruiective Services is responsible 
for: 

i. Reporting on the status of security, fire and rescue plans, train- 
ing, equipment, and personnel. 

. Summarizing the design and operational hazardous situations anu 
providing the status of open hazards in the Integrated Hazard 
Analysis. 

The Director, Biomedical Office is responsible for presenting an 
assessment of medical equipment, facilit es, one personnel witnin the 
Biomedical Office's area of responsibi 1 ity. 

The Manager, Cargo Projects Office and the Director of Cargo Operations 
are responsible for presenting their respective assessments of 
facilities, systems, and equipment including associated hardware 
(within their O&M responsibility) as defined by the Facilities, 

Systems, and Equipment Organizational Level OMD Baseline and 
essential to the cargo processing facility, GSE, and software. 

The Manager, Cargo Projects Office, and the Director of Cargo 
Operations are also responsible for presenting an assessment of 
responsibilities. The presentations shall include: 


. Open work and special tests to be completed (e.g., OMis, Category 
II paper. Operations & Maintenance Requirements/Speci f icatioris 
(UMRS ) not met. Engineering Support Requests (ESRs), an g concerns 
related to their successful completion). 

. Open or potential problems documented by problem report (s), devia- 
tiGn/wai ver(s) or equivalent report(s). The discussicr should 
loentify operational, concern, corrective action in work or 
planned, and impact on the launch schedule. 

. Status of the launch commit criteria arid a discussion of any 
changes (deltas) to. the previous mission baseline. 

. Identification and status of any essential facility*, system, or 
equipment not operational due to shortage of spare p arts, includ- 
ing discussion of any work-arounds. 

. A discussion of significant changes in the CDCT a. id launch count- 
down OMIs. 
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PERSONNEL AND RESPONSIBILITIES (cont'd) 

. Status of other operations and support functions, to include, but 
not be limited to the following: 

- Range support. 

- Personnel training. 

-Replacement of limited life components. 

-■ Material shortages. 

- Documentat ioni 

. Status of operational software. 

. A summary of all residual operational hazards. 

The Manager, Caryo Projects Office ana the Director of Cargo Operations 
are responsible for implementing the necessary reviews, certi f ications 
and assessments related to the OH cargo and elements thereof in pre- 
paration for the STS FRR. 
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PURPOSE 

The purpose of this plan is to define the activities for accom- 
plishing the Space Shuttle Program Orbital Flight Test (OFT) 
Design Certification Review (OCR). 

SCOPE 

This plan establishes schedules, responsibilities, and proce- 
dures for the organization and presentation of briefings and 
supporting data required for the OFT Design Certification Review. 

APPLICABILITY 

This plan is applicable to all NASA organizations and their con- 
tractors who have responsibility for the design and development 
of Space Shuttle System flight and ground hardware and software. 
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The Design Certification Review is a formal, comprehensive 
review of all hardware and software elements of the Space 
Shuttle System. The primary review will be conducted at 
the project level with summary briefings and certifications 

presented to program management and will culminate in final 
presentations to the OCR Board chaired by the Associate 

Administrator for Space Transportation Systems. 

The presentations must show: 

o Design requirements and end item specifications are consistent, 
o Design requirements have been met and confirmed by test 
results and/or analyses. 

o Appropriate remedial actions have been taken to resolve any 
significant problems that have arisen, 
o Open items and issues that mu st be resolved before flight? 

The oral briefings to the DCR Board will be supplemented by 
written statements by cognizant managers certifying that hardware 
and/or software elements for which they are responsible meet all 
of the requirements set forth in applicable specifications (or 
that approved waivers have been obtained) and are capable of 
supporting the Space Shuttle Orbital Flight Test Program. 
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DCR CYCLE - The DCR will be divided into three phases: 
o Phase I - Individual review of data and preparation of 
briefings by respective project managers 
o Phase II - Program Manager's review and critique of the summary 
briefings and supporting data 

o Phase III - Sunmary briefings with written supporting data 
to the DCR Board 

Phase I - Phase I will consist of reviews by individual project managers 
covering the respective program elements for which they have responsi- 
bility. Design assessment briefings will be prepared by the appropriate NASA/ 
contractor organizations and presented to the respective project managers for 
review. Corrective actions and revisions resulting from these reviev/s will 
be incorporated into the briefings prior to the Phase II review. 

Phase II - Phase II will consist of a preboard review of the 
presentations and supporting data on all program elements by the 
Program Manager. 

The purpose of the Phase II review is to: 

o Assure adequate depth, accuracy and consistency of coverage 
for each element of the Space Shuttle System . 
o Technically critique the data to be submitted, and the proposed 
presentations. 

o Assure that presentations and supporting data are properly integrated 
for submittal to the DCR Board. 

The presentations and supporting data for Phase II should 
approach the content of the briefings and data to be 
submitted during Phase III. Material added to, or revised in, the 
briefings on supporting data subsequent to the Phase II review should be 
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coordinated with the Program Manager prior to Phase III of the DCR. 
Changes and revisions should be made only as the result of: 
o Additional test results, solutions to problems, and reliability/ 
verification data, 
o Response to action items, 
o Open item closures. 

o Actions assigned by the Program Manager, 
o New problems and proposed corrective actions. 

Phase III - Phase III of the DCR will consist of the oral briefings 
and supporting data to the DCR Board established and chaired by 
the Associate Administrator for Space Transportation Systems. 
Post-Phase III - Subsequent to the Phase III review, actions 
assigned by the DCR Board must be completed by the responsible 
program elements. Recommended action closures will be submitted 
through the Program Manager and the Program Director to the 
Associate Administrator for recommended closure. All assigned 
actions must be closed prior to the Flight Readiness Review (FRR) 
for the applicable orbital flight. 
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Space Shuttle Program Director - is responsible for: 
o Organizing, scheduling and conducting Phase III of the 
DCR. 

o Reviewing proposed closeout of action items assigned 
during Phase III. 

o Preparing and executing a Space Shuttle Program Design 
Certification Document identifying any actions upon 
which certification of the Space Shuttle System is con- 
tingent. 

Manager, Space Shuttle Program - is responsible for: 
o Planning for the OFT DCR. 

o Organizing, scheduling and conducting Phase II of the DCR. 
o Establishing and chairing a Phase II DCR Pre-Board, 
o Reviewing and approving closeout of actions resulting from 
the Phase II review. 

o Reviewing and approving additions or revisions to the 
Phase II presentations and supporting data, 
o Reviewing proposed closeout of actions resulting from the 
Phase III review and submitting recommended closeouts to 
the Program Director and the Associate Administrator for 
Space Transportation Systems. 
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Manager for Management Integration - is responsible for: 
o Detailed planning for the OFT DCR. 

o Reviewing formats and element plans for the DCR briefings 
and supporting data. 

o Providing the status of DCR implementation progress to the 
Program Manager. 

Manager for Systems Integration - is responsible for: 
o Implementing the requirements of the OFT DCR Plan relative 
to Space Shuttle System Integration tasks, 
o Reviewing and critiquing the DCR Phase I Space Shuttle 
System Integration Plan. 

o Assuring the preparation and updating of the Space Shuttle 
System Integration briefing and supporting data, 
o Presenting the Space Shuttle System Integration assessment 
and certification at the Phase II and Phase III DCR Reviews. 
Project Managers - The project managers for the Orbiter, the 
Space Shuttle Main Engine, the External Tank, the Solid Rocket 
Booster, Crew Related Government-Furnished Equipment, Launch 
and Landing, and Integrated Communications and Data Systems 
are responsible for: 

o Implementing the requirements of the OFT DCR Plan within 
their respective projects. 

o Reviewing and critiquing the DCR Phase I Space Shuttle 
reports for their project elements, 
o Assuring the preparation and updating of the briefings and 
supporting data for the Phase II and Phase III DCR 
Reviews. 
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PERSONNEL AND RESPONSIBILITIES (cont’d) 
o Presenting the assessment and certification for their project 
elements at the Phase II and Phase III DCR Reviews. 

Director, Safety, Reliability and Quality Assurance 
(SR&QA) - The Director, SR&QA, is responsible for: 
o Presenting a summary of hardware/software Level II accepted 
risks; projects will present Level III accepted risks, 
o Reviewing and critiquing the DCR data presented at the 
Phase II and Phase III reviews. 


PHASE I REVIEW 

ORGANIZATION . SYSTEM/ELEMENT 

Sys. Int. 

Integr. Comm. & Data Sys. 
JSC GFE 

Orbiter 

Safety 


APPROVAL RESPONSIBILITIES 
Mgr. Sys. Int. 

Mgr. Integr. Comm. & Data Sys. 
Mgr. GFE Proj. 

Mgr. Orb. Proj. 

Director - SR&QA 


ET 

MSFC SSME 

SRB 


Mgr. ET 
Mgr. ET 
Mgr. SRB 


KSC Launch & Landing 


Mgr. Launch & Landing Proj. 
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PHASE II REVIEW 

Approval Responsibility - DCR Preboard 
Preboard membership is as follows: 

Chairman: JSC SS Program Mgr. 

Members: MSFC Shuttle Proj. Mgr. 

KSC Shuttle Proj. Mgr. 

JSC Dir. of Engineering and Development 

JSC Shuttle Payload Integration and Development Prog. Mgr. 

JSC Director of Flight Operations 

JSC Manager for OFT 

JSC Dir. of Data Systems and Analysis 

JSC Dir. of Safety, Reliability & Quality Assur. 

JSC Manager Program Operations 

JSC Dir. of Space and Life Sciences 

SAMSO-Deputy for Space Launch Systems 

NASA Hqs. Director, Space Shuttle Program (ex officio) 

JSC Director (ex officio) 

JSC Deputy Director (ex officio) 

PHASE III REVIEW 

Approval Responsibility - DCR Board 

Chaired by the Assoc. Admin, for Space Transportation Systems, 
membership will be established at a later date. 
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The Director of Office of Flight Assurance has overall responsibility for the 

Spacecraft Design Review Program (SDRP), and will appoint the members 
and Chairman of each Design Review Team (DRT) by memorandum. 

The Chief, Systems Review Office is responsible for implementing and 
executing design reviews and generating design review plans and pro- 
cedures . 


FEEDBACK AND CLOSEOUT 


At the completion of each review a formal report to the 

Director, will be prepared by the DRT. Minimum requirements 

of the report are: 

(a) a summary statement of the DRT findings; 

(b) recomnendations made by the DRT to the project; and 

(c) comments or responses of the project to the findings and 
recommendations of the DRT. 

The completed design review report will contain the results of 
each review conducted for the project together with a mission 
launch readiness statement issued by the Chairman of the DRT. 


The design review report will be issued and formally accepted 
by the Director prior to the launch operation. 


5 
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PURPOSE 


This instruction defines the policy and general procedures for the 
design review of projects. 


APPLICABILITY 


The provisions of this instruction are applicable to all spacecraft 
projects, including experiments and unique support equipment. 


DEFINITION 


Design Review is a systematic, technically oriented, and documented 
evaluation of spacecraft and associated equipment by a team of 
specialists. 
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All spacecraft and major flight experiments shall be subject to the 
SDRP. 

The SDRP shall be supported by all Directorates who will furnish the 
DRT with senior personnel experienced in the required technical 
disciplines. 


DESIGN REVIEW OBJECTIVES 


The primary objectives of the SDRP is to enhance the probability of success 
of spacecraft missions. This objective will be achieved by bringing 
to bear on each project the cumulative knowledge of a team of engineers 
and scientists who have had extensive prior experience with the particular 
types of systems and functions involved. While the design review is tech- 
nically oriented, proper consideration will be given to constraints oper- 
ating on the projects; particularly those Involving primary mission ob- 
jectives and program costs and schedules. These reviews shall assure that 
each project has the benefit of Centerwide experience gained on other pro- 
jects. They shaTl also provide the Center's review of the projects' Systems 
Safety Program. 
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The Design Review Team 

The DRT will include personnel experienced in design, systems engineering 
and integration, reliability, quality assurance, testing, materials, and 
other applicable disciplines. The personnel will be selected from 
throughout the Center with the approval of the appropriate Directors. 

Number of Reviews 

(1) The Chief, Systems Review Office, Office of Flight Assurance, 

in conjunction with the individual Project Manager will develop 
a total design review plan. Except in cases of repeat missions, 
the following reviews will normally be held: 

(a) Design Reviews - these reviews occur during the design phase 
and prior to the start of assembly. They will emphasize 
implementations of design approaches resulting from the study 
phase as well as test plans for the prototype and flight 
systems. For new systems generally two design reviews will 
be conducted. 

(b) Environmental Review - this review occurs after prototype 
qualification testing, or prior to acceptance testing if 
no prototype is used. The primary purpose of this review 
is to determine the qualification status of the hardware 
and to evaluate flight acceptance test plans. 

(c) Flight Readiness Review - this review will usually take 
place prior to shipment of the flight spacecraft to the 
launch range, and will concentrate on spacecraft per- 
formance during acceptance testing. 

(d) Flight Operational Readiness Review - this review will be 
conducted when a flight operations plan is available. While 
all of the previous reviews involve operations, this review 
will enphasize the final orbital operations plans* as well 
as the compatibility of the spacecraft with ground support 
equipment and ground network, including summary results of 
the network compatibility tests. 

(2) All flight experiments which are required for mission success 

are subject to this review program. One or two Experiment de- 
sign reviews, depending on need, shall be held prior to in- 
tegration. 

Design Review Schedule 

The several reviews will be conducted on a schedule determined by 

the Chief, Systems Review Office after consultation with the individual 

Project Manager. The major reviews shall be depicted in the 

Monthly Status Review. 
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APPLICATION OF MINIMUM STANDARD REVIEWS 
TO PAYLOAD CLASSIFICATIONS 


The issue of the applicability of the six identified minimum standard 
reviews to the payload classifications defined in NMI 8010.1 was addressed 
based on the data available. Only one of the documents reviewed addressed 
the identification of the nature of the reviews or the requirements for 
data evaluation as a function of payload classification. Therefore, the 
guidelines contained in this appendix are the result of limited input from 
NASA documents combined with interpretation and judgement by ARINC 
Research based on the definitions of the payload classifications. 
Consequently, these review requirements should be reviewed with the 
individual centers to obtain more NASA input. 

Due to the nature of Class D payloads, where formal design 
verification requirements are limited to safety and STS-compatibility 
verification, no attempt was made to evaluate the applicability of 
different reviews or data items. In practice, most of these payloads will 
be provided to NASA by private companies or universities (such as Getaway 
Special canisters), who will perform the majority of the reviews 
internally. Reviews by NASA will be limited to safety and interface 
analyses, with limited formal evaluation by NASA of their performance 
characteristics. In general, the requirements for the evaluation of 
Class C payloads could be followed for a Class D payload where applicable 
and cost-effective. 

The classification of the six minimum reviews as a function of 
payload classifications A, B and C is shown in Table E-l. 

Most of the reviews are formal reviews, which have the following 
characteristics: 

- The convening authority issues a written charter for the review 
which identifies the scope and objective of the review. The 
charter delineates the Board members, review agenda, location, 
timetable for the review activities, and specific areas the board 
will address. 

- The review board consists of the board chairman, board secretary 
and an appropriate number of personnel with the proper 
qualifications to address the issues before the Board. 
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Table E-l. CLASSIFICATION OF REVIEWS 

Review 

Class A 

Class B 

Class C 

Preliminary Requirements Review 

Formal 

Formal 

Informal 

Preliminary Design Review 

Formal 

Formal 

Informal 

Critical Design Review 

Formal 

Formal 

Informal 

Preshipment Review 

Formal 

Formal 

Informal 

Design Certification 

Formal 

Formal 

Formal 

Flight Readiness Review 

Formal 

Formal 

Formal 


- Review presentation is generally in the form of viewgraphs, with 
copies provided to Review Board members and other attendees. 

- Requests for action are submitted by Review Board members (and 
other attendees if invited to participate) on forms provided by 
the Board secretary. Following the review, the review board 
reviews the actions items, consolidates them and ensures that the 
chairman and cognizant manager understand the intent of the 
requests. These requests for action are normally included as an 
attachment to the board report . 

- Following appropriate Board deliberation and consensus where 
possible, the findings of the Board and any recommendations are 
prepared by the secretary, approved by the Board, and sent to the 
convening authority and the cognizant manager for their study and 
disposition. Dissenting opinions may also be attached. The 
cognizant manager determines disposition of the findings and 
recommendations; e.g., identifies action items, formulates an ad 
hoc study team, makes policy decisions, or takes whatever action 
he believes is appropriate for each specific finding, concern, or 
recomraendat ion . 

- The responsibility for accepting, rejecting, or acting in part on 
the Board's recommendations rests with the cognizant manager. A 
written response to the board provides documentation of the 
disposition of the recommendations and provides assurance that the 
Board's recommendations are properly interpreted and considered. 
The cognizant manager reviews proposed action items with those to 
whom the action is assigned. They should establish an action plan 
(i.e., how the action item will be resolved, by whom, when). The 
cognizant manager should report on the status of the action items 
to the convening authority, with copies to the Board secretary and 
the Office of Engineering and Review. 
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- Action items ate closed by memorandum, and copies ace sent to the 
chairman for approval and distribution. The memoranda are to 
include the supporting information for the recommended closure 
action. The secretary of the Board maintains a file of all Board 
actions, including closure memoranda, as a central point for all 
Board members. 

A number of the reviews for Class C payloads may be informal in 
nature. An informal review is distinguished from a formal review in that: 

- A memorandum identifying the review subject, chairman, independent 
reviewer (s), review time and location replaces a formal written 
charter for the review. 

- The Review Board may consist of a Board chairman and a minimum 
number of individuals, consistent with the size of the program, 
not associated with the program or project under review. 

- Review presentation material may be limited to existing 
documentation, rather than more formal viewgraphs or other 
specially-prepared material. 

- Requests for action during the review may be oral or written but 
special forms are not required. 

- A brief memo may be used to document the review findings rather 
than a formal report. 

- Levying the closure of action items may be accomplished by the 
cognizant manager rather than through formal memoranda to the 
Review Board. 

- At preestablished intervals the cognizant manager should update 
the listing of action item closures and obtain new closure dates 
on past-due items. This information is distributed periodically 
until all action items are closed. 

Other aspects of an informal review are similar to a formal review. 

The last reviews for a Class C payload, the DCR and the FRR, should 
be formal reviews to provide the opportunity for more formal review and 
documentation of the project or program prior to flight commitment. It 
was felt that informal reviews were not appropriate in the final stages of 
the system development. 

The applicability of the evaluation of specific data items as a 
function of payload classification during the six reviews is shown in 
Tables E-2 through E-7. No distinction could be made between A and B 
payloads for these minimum review items. The evaluation of certain data 
items are either not required for Class C payloads, limited to top-level 
analysis, or limited to safety and STS interface requirements only. 


E-3 



Table E-2. PRELIMINARY REQUIREMENTS REVIEW DATA 
EVALUATION APPLICABILITY 


Review Item/Data 


Class A 


Class B 


Class C 


- Statement of mission objectives 

- Mission requirements analyses 

- Preliminary preferred requirements 

- Functional analyses 

- Preliminary interface requirements 

- Support agreements 

- Preliminary reliability and quality 
assurance requirements 

- Scientific requirements 

- Concept drawings and sketches 

- Block and logic diagrams 


X 

X 

X 

X 

X 

X 

X 

X 

X 

X 


X 

X 

X 

X 

X 

X 

X 


X 

X 

X 

X 

X 

X 

X 


X 

X 

X 


X 

X 

Top-Level 

only 


- Costs and schedules 

- Design concept documentation 

- General test requirements 

- Preliminary study reports 

- Heritage analyses 


X 

X 

X 

X 

X 


X 

X 

X 

X 

X 


X 

X 

X 

X 
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Table E-3. PRELIMINARY DESIGN REVIEW DATA EVALUATION APPLICABILITY 


Review Item/Data 


Class A Class B Class C 


- Status of PRR action items 

- Final functional requirements 

- Functional requirements/- 
specifications compliance 
analyses 

- inherited designs and hardware 
suitability analyses 

- Environmental design requirements 

- Interface design requirements 

- Design parameters, restraints, 
and constraints 

- Block diagrams 

- Single-point failure analyses 

- Preliminary parts, materials, 
and processes list 

- Hardware and software development 
and schedule (including verifica- 
tion tests on analyses to be 
performed) 

- Software standards to be applied 


- Experiment performance analysis 



Safety and 
STS interface 
items only 


















Table E-3. PRELIMINARY DESIGN REVIEW DATA EVALUATION APPLICABILITY 

(continued) 


Review Item/Data 

Class A 

Class B 

Class C 

- Preliminary data management flow 

X 

X 

X 

and reduction plans 




- Safety, reliability, and quality 

X 

X 

X 

assurance compliance plans 




- Spares provisioning plans 

X 

X 

X 

- Support equipment requirements 

X 

X 

X 

and plans 




- Preliminary operations planning 

X 

X 

X 

- Schedules and status of the 




project including cost as well 
as technical developments 

X 

X 

X 

- Design optimization analyses 

X 

X 

X 

- Plans and controls documentation 

X 

X 

X 

- Design traceability 

... . 

X 

X 
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Table E-4. CRITICAL DESIGN REVIEW DATA EVALUATION APPLICABILITY 


Review Item/Data 

- Status of PDR action items 

- Hardware and software inheritance 

- Specifications (test and 
performance) 

- Prototype test results 


Class A 


Class B 


X 

X 

X 


X 

X 

X 


Class C 
X 

X 

X 


- Design mechanization and analysis | X 

- Design trade-offs and I X 

alternatives considered 

- Detailed interfaces and cable 
configurations 

- Detailed analysis of failure modes | X 

- Safety analyses and human factors | X 

- Maintainability, repair ability, I X 

operability, and reliability 


X 

X 


X 

X 

X 


STS interfaces 
only 


- Schedule and resource plans 


X 


X 


X 


- Man-machine interfaces 


X 


X 


STS interfaces 
only 


- Training plans 


X 


X 


X 


- Conformance of the design to the 
functional requirements 


X 


X 


X 


- Differences between the system 
and subsystem performances 
relative to the performances 
estimated at the preliminary 
design review 


X 


X 


X 


- Detailed design parameters, 
restraints, and constraints 


X 


X 


X 
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Table E-4. CRITICAL DESIGN REVIEW DATA EVALUATION APPLICABILITY 

(continued) 


Review Item/Data 


Class A 


Class B 


Class C 


Interface details/agreements 
Weight and power 
Detail circuit drawings 


X 

X 

X 


X 

X 

X 


X 

X 


- Electronic parts classification 

- Screening specifications 

- Nonelectronic parts, materials, 
and processing list 

- Purchased devices lists 


X 

X 

X 

X 


- Materials and processing 
specifications 

- Identification and traceability of 
materials and specifications 

- Electronic and mechanical parts 
stress analysis results 

- Worst-case analysis of end-item 
failure 

- Conformance to environmental 
design requirements 

- Details of design, construction, 
and electronics packaging 

- Quality assurance plans and 
procedures 


X 

X 

X 

X 

X 

X 

X 


X 

X 

X 

X 

X 


X 

X 


Lot trace- 
ability 


X 


X 


X 


X 


X 


X 


X 


- Manufacturing plans 


X 


X 


- Configuration control plans 
(hardware and software) 


X 


X 


X 














Table E-4. CRITICAL DESIGN REVIEW DATA EVALUATION APPLICABILITY 

(continued) 

Review Item/Data 

Class A 

Class B 

Class C 

- Documentation status 

X 

X 

X 

- Flow plan and schedule status 

X 

X 

X 

(hardware and software) 




- Problem/ failure reporting system 

X 

X 

X 

- Failure modes and effects analysis 

X 

X 


- Preliminary test plans 

X 

X 

X 

- Developmental and detailed 

X 

X 

X 

test results 




- Qualification and acceptance 
test plans 

X 

X 

X 

- Calibration plan 

X 

X 

X 

- Data management plan 

X 

X 

X 

- Formal and informal report plans 

X 

X 


- Final design verifications plans 

X 

X 

X 

- Initial user handling or operating 

X 

X 

X 

constraints and precautions 




- Spares provisioning 

X 

X 

X 

- Support equipment requirements 

X 

X 

X 

- Flight operations approach 

X 

X 

X 

- Schedule and cost status 

X 

X 

X 

- Risk assessment 

X 

X 

X 
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Table E-5. PRESHIPHENT REVIEW DATA EVALUATION APPLICABILITY 


Review Item/Data 


Class A 


Class B 


Class C 


- Status of prior action item 
closeouts 


X 


X 


X 


- Current approved drawings and 
specifications, and adequate 
definition of the end-item, 
including approved specification 
changes and waivers 


X 


X 


Top assembly 
drawings 
and 

definition of 
end- item 


- End-item status and documenta- 
tion, including all interface 
documentation, to assure that the 
latest changes have been incorpo- 
rated in affected hardware 


X 


X 


X 


- Test documentation status and 

determination (by review of the X 

test results) of the adequate and 
proper testing of end-items 

- Problem/Failure Report documenta- 

tion to determine its status and 
evaluate its total impact, X 

particularly with respect to risk 

for the mission 


X 


X 


X 


Safety and STS 
interface items 
only 


- Quality assurance history documen- 
tation to ensure that necessary 
inspections have been performed 
and all discrepancies have been 
properly closed out 

- Shipping and handling require- 
ments and plans 

- Safety compliance verification 

- Status and acceptability of the 
deliverable end-item data package 


X 


X 

X 

X 


X 


X 

X 

X 


X 


X 

X 

X 


- Risk assessment 


X 














Table E-6. DESIGN CERTIFICATION REVIEW DATA 

EVALUATION 

APPLICABILITY 

Review Item/Data 

Class A 

Class B 

Class C 

- Status of previous action items 

X 

X 

X 

- Item change data 

X 

X 

X 

- Analysis and test documentation 

X 

X 


- Reliability documentation 

X 

X 

X 

- Safety analyses and human factors 

X 

X 

X 

- Specified requirements derived 

X 

X 

X 

from mission objectives 




- previous failures/ incidents 

X 

X 

X 

analysis 




- Physical and functional 
configuration verification 

X 

X 

Safety and STS 
interface items 

- Validation of analytical models 

X 

X 

only 

of equipment and system 




- Validation of thermal control 

X 

X 


design approach 




- Hazards control review 

X 

X 

X 

- Safety system and subsystem 

X 

X 

X 

checklist 




- Training of ground flight 

X 

X 

X 

personnel 




- Operational plans and status 

X 

X 

X 

- Mission rules 

X 

X 

X 
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Table E-6. DESIGN CERTIFICATION REVIEW DATA 

(continued) 

EVALUATION 

APPLICABILITY 

Review Item/Data 

Class A 

Class B 

Class C 

- Results and status of verification 




planning, testing and analysis for 
the total system and associated 
ground support systems 

X 

X 

X 

- Deviations from baseline 
configuration 

X 

X 

X 

- Configuration inspection 
documentation 

X 

X 

X 


i 
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Table E-7. FLIGHT READINESS REVIEW DATA EVALUATION APPLICABILITY 


Review Item/Data 

Class A 

Class B 

Class C 

- Status of prior action item closeouts 

- Status of vehicle and launch support 

X 

X 

X 

facilities, systems, equipment, range 
support, launch commit criteria and 
personnel training 

X 

X 

X 

- Residual hazards and problem reports 

X 

X 

X 

- Status of operation/maintenance 
instructions 

X 

X 

X 

- Configuration changes since prior 
review 

X 

X 

X 

- Hardware and software anomalies 

X 

X 

X 

- Status of development and qualifica- 
tion testing 

X 

X 

X 

- Limited life components, life 
remaining, and age life/ time cycle 

X 

X 


- Status of logistics readiness 
(spares support) 

X 

X 


- Verification of documentation 
completion 

X 

X 

X 

- Identification of actual hardware 
shortages and open work items 

X 

X 

X 

- Safety, reliability, and quality 
assurance assessment 

X 

X 

X 

- Identification of problems that may 
constrain the flight 

X 

X 

X 

- Failures/incidents and accidents 
reports 

X 

X 

X 












